Media data decoding device

ABSTRACT

The present invention relates to media data decoding devices. Embodiments of the present invention pertain to devices that receive scalable media, receive scalable profile data, and generate rendered media data based on the scalable media data and the scalable profile data.

TECHNICAL FIELD

Embodiments of the present invention relate decoding devices and morespecifically to decoding devices that can decode scalable media datawithout knowledge of the encoding scheme of the media data.

BACKGROUND ART

Data delivery systems present many challenges for the system designer.For instance, clients can have different display, power, communication,and computational capabilities. In addition, communication links in thesystem can have different maximum bandwidths, quality levels, andtime-varying characteristics. A successful data delivery system shouldbe able to deliver data streams to a multitude of diverse clients overheterogeneous networks with time-varying characteristics.

Providing proper security in order to protect content from eavesdroppersis another important consideration in the design of data deliverysystems. Generally, to provide security, data are transported inencrypted form.

Intermediate nodes in the system may be used to perform streamadaptation, or transcoding, to scale data streams for differentdownstream client capabilities and network conditions. A transcodertakes a compressed, or encoded, data stream as an input, and thenprocesses it to produce another encoded data stream as an output.Examples of transcoding operations include bit rate reduction, rateshaping, spatial downsampling, and frame rate reduction. Transcoding canimprove system scalability and efficiency, for example, by adapting thespatial resolution of an image to a particular client's displaycapabilities or by dynamically adjusting the bit rate of a data streamto match a network channel's time-varying characteristics.

Intermediate nodes can collect and update information about local anddownstream network conditions and downstream client capabilities, andthen scale the data according to that information. This can be moreefficient than scaling the data at the source, because it is moredifficult for the source to collect up-to-date and detailed informationabout conditions inside the network, especially at locations in thenetwork relatively far removed from the source. Also, the sourceprovides only one control point at the beginning of the delivery path,while intermediate transcoding nodes provide many additional controlpoints at more strategic locations along the delivery path.

While network transcoding facilitates scalability in data deliverysystems, it also presents a number of challenges. The process oftranscoding can place a substantial computational load on transcodingnodes. While computationally efficient transcoding algorithms have beendeveloped, they may not be well-suited for processing hundreds orthousands of streams at intermediate network nodes.

Furthermore, transcoding poses a threat to the security of the deliverysystem because conventional transcoding operations generally requirethat an encrypted stream be decrypted before transcoding. The transcodedresult is re-encrypted but is decrypted at the next transcoder. Eachtranscoder thus presents a possible breach in the security of thesystem. This is not an acceptable situation when end-to-end security isrequired.

Compression, or encoding, techniques are used to reduce the redundantinformation in data, thereby facilitating the storage and distributionof the data by, in effect, reducing the quantity of data. The JPEG(Joint Photographic Experts Group) standard describes one popular,contemporary scheme for encoding image data. While JPEG is satisfactoryin many respects, it has its limitations when it comes to current needs.A newer standard, the JPEG2000 standard, is being developed to meetthose needs. However, even with the JPEG2000 standard, decryption ofencrypted data is needed for transcoding, and transcoding processesremain computationally intensive. Furthermore, the introduction of a newstandard such as JPEG2000 means that each of the large number of networknodes, as well as client devices, needs to be updated in order to bemade compliant with the JPEG2000 standard.

Accordingly, a method and/or system that can allow scaling (e.g.,transcoding) of data in a secure and computationally efficient mannerwould be advantageous. A system and/or method that can accomplish thoseobjectives on legacy devices would be more advantageous. The presentinvention provides these as well as other advantages.

DISCLOSURE OF THE INVENTION

Embodiments of the present invention pertain to devices that receivescalable media, receive scalable profile data, and generate renderedmedia data based on the scalable media data and the scalable profiledata.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthis specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention:

FIG. 1 is an example of a bit stream including data according to oneembodiment of the present invention.

FIG. 2 illustrates examples of data segments in a bit stream accordingto one embodiment of the present invention.

FIG. 3 is a block diagram showing functional elements in a simplifieddata delivery system according to one embodiment of the presentinvention.

FIG. 4A shows information flowing into and out of an encoder accordingto one embodiment of the present invention.

FIG. 4B shows information flowing into and out of an encoder accordingto a second embodiment of the present invention.

FIG. 5A shows information flowing into and out of a data scaleraccording to one embodiment of the present invention.

FIG. 5B shows information flowing into and out of a data scaleraccording to a second embodiment of the present invention.

FIG. 6 is a flowchart of a process for scaling (e.g., transcoding) dataaccording to one embodiment of the present invention.

FIG. 7 is a flowchart of a process for encoding data according to oneembodiment of the present invention.

FIG. 8 is a flowchart of a process for decoding data according to oneembodiment of the present invention.

FIG. 9A shows a system for coupling data with a scalable media accordingto one embodiment of the present invention.

FIG. 9B shows a system for coupling data with a scalable media accordingto one embodiment of the present invention.

FIG. 10 shows information flowing into and out of a data scaleraccording to a second embodiment of the present invention.

FIG. 11 shows the steps performed in a method for associating data withportions of a scalable media according to one embodiment of the presentinvention.

FIG. 12 is a flowchart of the steps performed in a method of scalingencrypted scalable media according to one embodiment of the presentinvention.

FIG. 13 is a flowchart of the steps performed in a method of decodingscalable media according to one embodiment of the present invention.

FIG. 14 illustrates a method for encoding a progressively encryptedsequence of scalable data according to one embodiment of the presentinvention.

FIG. 15A illustrates the transcoding by truncation of a progressivelyencrypted sequence of scalable data according to one embodiment of thepresent invention.

FIG. 15B illustrates the transcoding by truncation of a progressivelyencrypted sequence of scalable data according to an alternate embodimentof the present invention.

FIG. 16A illustrates the transcoding by truncation of a progressivelyencrypted sequence of scalable data according to one embodiment of thepresent invention.

FIG. 16B illustrates the transcoding by truncation of a progressivelyencrypted sequence of scalable data according to one embodiment of thepresent invention.

FIG. 17 is a flowchart of the steps performed in a method for encoding aprogressively encrypted sequence of scalable data according to oneembodiment.

FIG. 18 is a flowchart of the steps performed in a method of transcodinga progressively encrypted sequence of scalable data according to oneembodiment.

FIG. 19 is a flowchart of the steps performed in a method of decoding aprogressively encrypted sequence of scalable data according to oneembodiment of the present invention.

FIGS. 20, 21 and 22 are block diagrams of encoders, according toembodiments of the present invention.

FIGS. 23, 24 and 25 are block diagrams depicting decoders, according toembodiments of the present invention.

FIGS. 26, 27, 28, 29, 30, 31 and 32 are block diagrams depictingtranscoders, according to embodiments of the present invention.

The drawings referred to in this description should not be understood asbeing drawn to scale except if specifically noted.

BEST MODE FOR CARRYING OUT THE INVENTION

Reference will now be made in detail to various embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. While the invention will be described in conjunction withthese embodiments, it will be understood that they are not intended tolimit the invention to these embodiments. On the contrary, the inventionis intended to cover alternatives, modifications and equivalents, whichmay be included within the spirit and scope of the invention as definedby the appended claims. Furthermore, in the following description of thepresent invention, numerous specific details are set forth in order toprovide a thorough understanding of the present invention. In otherinstances, well-known methods, procedures, components, and circuits havenot been described in detail as not to unnecessarily obscure aspects ofthe present invention.

Overview

Embodiments in accordance with the present invention are discussedprimarily in the context of digital image data, in particular for stillimages. Digital image data can result from “real world” capture using adigital camera, for example. Digital image data can also becomputer-generated using, for example, a paint program, screen capture,or the conversion of a graphic into a bitmap image. However, the presentinvention is not limited to digital image data. Instead, embodiments ofthe present invention are well suited for use with speech-based data,audio-based data, video-based data, web page-based data, graphic data,text-based data (e.g., electronic documents), and the like.

Furthermore, embodiments in accordance with the present invention aredescribed for data that are scalably encoded using an encoding schemecompliant with, or substantially compliant with, the JPEG2000 standard.However, the present invention is not so limited. In general,embodiments according to the present invention are directed toward anydata that can be scalably encoded and, specifically, any data thatcombines scalable encoding with progressive encryption.

For purposes of the present application, scalable encoding is defined asa process which takes original data as input and creates scalablyencoded data as output, where the scalably encoded data has the propertythat portions of it can be used to reconstruct the original data atdifferent levels of quality, resolution and the like. Specifically, thescalably encoded data is often thought of as an embedded bit stream. Aportion of the bit stream can be used to decode a baseline-qualityreconstruction of the original data, without requiring any informationfrom other portions of the bit stream. Progressively larger portions ofthe bit stream can be used to decode improved reconstructions of theoriginal data. JPEG2000 is one example of a scalable encoding scheme.Other scalable encoding schemes that may be used in accordance withembodiments of the present invention include, but are not limited to, 3Dsubband coding and MPEG-4 FGS (Moving Picture Experts Group-4 FineGranularity Scalability).

For purposes of the present application, progressive encryption isdefined as a process which takes original data (plain text) as input andcreates progressively encrypted data (cipher text) as output, where theprogressively encrypted data has the property that a first portion ofthe encrypted data can be decrypted alone, without requiring informationfrom the remainder of the original data. Progressively larger portionscan be decrypted with this same property, in which decryption canrequire data from earlier but not later portions of the bit stream.Progressive encryption techniques that may be used in accordance withthe present invention include, but are not limited to, popularencryption primitives such as the Data Encryption Standard (DES),Triple-Des (3DES), and the Advanced Encryption Standard (AES).

General Discussion of JPEG2000 Encoding

In general, a number of stages constitute an encoding process compliantwith the JPEG2000 standard. These stages are referred to herein as 1)preprocessing; 2) discrete wavelet transformation; 3) quantization; 4)embedded block coding; 5) rate control; and 6) bit stream organization.Embodiments of the present invention are not limited to these stages,and are also not limited to the particulars of the JPEG2000 standarddescribed below. In fact, as will be seen, embodiments of the presentinvention introduce features beyond the syntax of the JPEG2000 standard,resulting in scaling (e.g., transcoding) processes that are morecomputationally efficient, and allowing scaling to be performed bydevices that perhaps are not aware of the JPEG2000 standard.

JPEG2000 allows for both lossless and lossy compression; the focusherein is on lossy compression. During preprocessing, tiling can beoptionally performed to partition an original image into tiles. Alsoduring preprocessing, the input data may be adjusted so that the datahas a nominal dynamic range centered on the value zero. Finally duringpreprocessing, color data may be transformed into Y, C_(r) and C_(b)color components.

During discrete wavelet transformation, image tiles can be decomposedinto high and low subbands. Tiles can also be partitioned intocode-blocks (e.g., 64×64 or 32×32 samples). More specifically, eachsubband is divided into rectangular blocks called “precincts,” and eachprecinct is further divided into non-overlapping rectangles calledcode-blocks.

The wavelet coefficients are quantized during the quantization stage. Inthe embedded block coding stage, each code-block is encoded separately.Rate control is a process by which the encoded bit stream is altered sothat a target bit rate can be reached. Each encoded code-block may bereviewed to determine the extent to which it can be truncated in orderto achieve the target bit rate.

In bit stream organization, encoded data are separated into what arereferred to in the JPEG2000 standard as “packets” (the contents of aJPEG2000 packet are described by the JPEG2000 standard). The packets arethen multiplexed together in an ordered manner into a bit stream. Notethat the use of the term “packets” according to JPEG2000 is generallydifferent from the more conventional use of that term. That is, JPEG2000packets are multiplexed into a bit stream, which may then be packetizedinto data packets that are sent over a network, for example.

According to JPEG2000, the order of the data (e.g., the JPEG2000packets) in the bit stream is referred to as a “progression.” There area number of different ways to order the packets, such asprecinct-component-resolution-quality orresolution-quality-component-precinct. According to JPEG2000, “quality”may instead be referred to as “layer.” These terms are known to thosewho are familiar with the JPEG2000 standard.

The order of the data in the bit stream may extend through the length ofthe bit stream. Alternatively, the order of the data in the bit streammay change to a different order at some point in the bit stream. Theorder of the data at any particular point in the bit stream is not ofsignificance to the discussion herein; what is of significance is thatthe data are in a particular order that is prescribed by the encodingscheme, in this case an encoding scheme compliant or substantiallycompliant with the JPEG2000 standard.

FIG. 1 is an example of a JPEG2000 bit stream 10 including a header 11and encoded data 12 according to one embodiment of the presentinvention. Header 11 includes a start of code stream marker SOC, animage and tile size marker SIZ, a coding style default marker COD, aquantization default marker QCD, a start of tile-part marker SOT, and astart of data marker SOD. Bit stream 10 also includes an end of codestream marker EOC. The functions performed by each of these markers isunderstood by those familiar with the JPEG2000 standard.

In the example of FIG. 1, the encoded data 12 is ordered according to aprecinct-component-resolution progression; however, as presented above,different progressions can be used. In the example of FIG. 1, there aren precincts (P0, P1, . . . , Pn), three components (C0, C1, and C2), andthree resolutions (R0, R3 and R2). (For simplicity of illustration,quality is not included in the progression.) Encoded data 12 is orderedfrom most significant to least significant. In the example of FIG. 1,the encoded data 12 is ordered first by precinct, then by component,then by resolution.

To decode a bit stream of encoded data, a process that is in essence theopposite of the encoder process discussed above is employed. The encodedbit stream can be decoded in many different ways, depending on, forexample, the capabilities of the displaying device and other practicalconsiderations, as well as the specific interests of the viewer (e.g.,one use may be satisfied with a low resolution image, while another maydesire a high resolution image). According to JPEG2000, it is possibleto locate, extract and decode the data required for the desired imageproduct, without decoding the entire bit stream. However, the bits thatcan be conventionally extracted are limited by the syntax of JPEG2000.Also, to extract those bits, knowledge of the scheme used to encode thedata (e.g., a JPEG2000-compliant scheme) is also required. For instance,to find the bits to extract, a device needs to be aware of JPEG2000 inorder to read the bit stream 10. As will be seen, embodiments of thepresent invention allow bits to be extracted beyond the syntax ofJPEG2000 and without requiring knowledge of the encoding scheme.

Scaling Encoded Data Without Requiring Knowledge of the Encoding Scheme

In the example of FIG. 1, the encoded data 12 includes a sequence of Nbits, labeled 0 through N. According to embodiments of the presentinvention, different segments of bits are identified within the sequenceof N bits. These segments are identified in FIG. 1 as data segments 13,14 and 15. As illustrated by FIG. 1, the data segments may overlap eachother in part or in entirety. Also, there can be data segments that donot overlap any other data segments. For example, segments 13 and 14overlap, while segment 15 does not overlap either segment 13 or 14.

Importantly, the segments 13, 14 and 15 are not constrained by thesyntax of JPEG2000. That is, the beginning and ending points of a datasegment are independent of the format of bit stream 10 that is dictatedby JPEG2000. Thus, for example, segment 13 extends from the portion ofbit stream 10 that includes C0 into the portion that includes C1. Also,segment 13 begins at a point in the midst of the P0-C0-R1 portion andends in the midst of the P0-C1-R2 portion. Furthermore, as will be seen,the locations of the data segments 13, 14 and 15 in bit stream 10 can bedetermined without reading bit stream 10, including the information inheader 11.

In general, a data segment according to the present invention (e.g.,data segments 13, 14 and 15) can begin and end anywhere in the bitstream 10, and segments can overlap each other. Any number of such datasegments can be identified, and a segment can have any length withinencoded data 12. Thus, a particular bit or sequence of bits can be amember of more than one data segment. The segments do not necessarilyencompass the entire length of the encoded data 12. That is, there maybe one or more portions of encoded data 12 that are not included in adata segment defined according to the present invention.

Furthermore, data segments defined according to the present inventionhave these important characteristics: they are independently decodable;they are independently encryptable and decryptable; and they areindependently checkable. That is, each data segment can be decodedindependent of any other segment. Similarly, each data segment can beencrypted and decrypted independent of any other segment. Also, achecksum, for example, can be applied to each data segment independentof any other segment.

FIG. 2 illustrates examples of data segments in a bit stream accordingto one embodiment of the present invention. As just described, encodeddata 12 includes N bits that are ordered and formatted depending on theencoding scheme. According to embodiments of the present invention, anumber of segments are identified within the encoded data 12.

In the example of FIG. 2, encoded data 12 includes bits identified asb1, b2 and b3, and other bits identified as r1, r2 and r3 (these bitsall lie within encoded data 12; however, they are separately illustratedfor clarity). Thus, for example, a segment can be defined as extendingfrom bit 0 to bit b1, another segment can be defined as extending frombit b1 to bit b2, and so on. A segment can instead be identified asbeginning at a certain bit number and having a certain length (measuredin bits). For example, a data segment can be defined as beginning at bitb0 and having a length equal to b1 bits, another segment can beidentified as beginning at bit b1 and having a length of (b2-b1) bits,and so on. Furthermore, a segment can extend from bit b1 to bit b3, orfrom bit b3 to N, and so on. That is, as mentioned above, a segment canhave any length, beginning at any bit and ending at any other bit, andtwo or more segments can overlap. For example, the data segment definedby bits r2 to r3 overlaps the data segment defined by bits b2 to b3.Segments may or may be of different lengths.

Importantly, the data segments defined according to the presentinvention and exemplified by FIG. 2 have starting and ending points thatare independent of the format specified by an encoding scheme forencoded data 12. As will be seen, the data segments defined according tothe present invention allow data, particularly encoded data, to beextracted or parsed from bit stream 10 without requiring knowledge ofthe encoding scheme that was used to encode the data. Moreover, the datacan be extracted from the bit stream 10 (FIG. 1) without decrypting thedata if the data are encrypted.

The capability to extract/parse data in this manner can beadvantageously applied to data scaling (e.g., transcoding) and decoding(decompressing), as will be seen. For example, to achieve a reduction inbit rate, the two data segments ranging from bits 0 to b1 and b2 to b3may be extracted from bit stream 10 as part of a transcoding operation.Then, those two segments can be combined in a scaled version of theencoded data and sent over the data delivery system. To achieve areduction in frame rate, for example, one or more data segments (e.g.,the data segment ranging from r1 to r2) are similarly chosen. Thus, thesingle set of data 12, ranging from bits 0 to N, can be organized intodata segments that allow scaling to be performed for a number ofdifferent scalable attributes (e.g., bit rate, frame rate, etc.). Forexample, the single set of data 12 can be organized into a first set ofdata segments that pertain to bit rate reduction, another set of datasegments that pertain to frame rate reduction, and so on. The set ofdata segments chosen for bit rate reduction may or may not include someportion of the set of data segments chosen for frame rate reduction, andvice versa.

Importantly, in the above examples, the segments selected for bit ratereduction, frame rate reduction, or the like are intelligently selectedduring encoding to achieve the desired scaling result while minimizingto a practical extent the impact on the image product. It is alsoimportant to note that a much finer granularity in data segment lengthscan be achieved than that illustrated by FIG. 2. As a result, scalingoperations can be performed to finer degrees.

FIG. 3 is a block diagram showing functional elements in a simplifieddata delivery system 30 according to one embodiment of the presentinvention. In the example of FIG. 3, system 30 includes an encoder 32, atranscoder 34, a first decoder 36 and a second decoder 38. System 30 maybe part of a larger system or network that includes similar functionalelements as well as other types of functional elements (e.g., storageelements, packetizers, streaming elements and the like). In the presentembodiment, encoder 32 encodes (compresses) data, transcoder 34transcodes (e.g., scales) encoded data, and decoders 36 and 38 decode(decompress) data. These functionalities may be performed in a singledevice or distributed among one or more devices that may be connected insome type of network.

Also, the elements 32, 34, 36 and 38 may perform functions other thanthose just described. For example, encoder 32 may also encrypt data andcalculate checksums or cryptographic checksums for checking the data,encoder 32 may also scale encoded data before sending the encoded datato another block, and decoder 38 may scale encoded data before decodingit. In one embodiment, encoder 32 uses an encoding scheme based on theJPEG2000 standard.

In addition, transcoder 34 may instead send a scaled version of encodeddata to another transcoder, which in turn scales the scaled data andsends it to another transcoder, and so on. Also, even though decoder 36receives scaled data from transcoder 34, decoder 36 may further scalethe scaled data. Furthermore, the decoder 36 and 38 may not be theend-user device. For example, decoder 36 and 38 may decode encoded data,and send the decoded data to, for example, a mobile phone that rendersand displays the image product.

Moreover, the data scaling function may be performed by a storage deviceor driver. For example, a storage device (e.g., a disk drive or DVDplayer) may scale the encoded data, passing only the appropriate datasegments to the actual decoder. In this manner, resources or time arenot wasted sending unnecessary information. This can also simplify thedecoder, because the decoder does not need to perform this processing.

FIG. 4A shows information flowing into and out of an encoder 32according to one embodiment of the present invention. In the presentembodiment, encoder 32 receives input data (e.g., data such as imagedata that are to be encoded). Encoder 32 also receives values ofscalable attributes.

For purposes of the present application, a scalable attribute is definedas an attribute having a range of values that specifies how the encodeddata are to be subsequently scaled. For image data, a scalable attributecan be, for example, bit rate, frame rate, resolution, color or blackand white, region of interest, tile, quality, image objects, orforeground versus background.

Scalable attribute values are values specified for the scalableattributes. For example, scalable attribute values for bit rate caninclude full bit rate (B), one-half bit rate (B/2), one-fourth bit rate(B/4), and so on.

Scalable attributes and values can be similarly defined for other typesof data (e.g., audio data, graphics data, and the like). For example, ascalable attribute value for audio data can indicate whether an audiotrack is stereo or mono. Electronic documents and text-based data mayalso be organized via scalable attributes that describe the content ofthe document or text-based data (e.g., chapters, sections, images,graphs, index, appendices, associated software, extra audiovisualmaterial, etc.). It is beneficial to be able to adapt the selection ofcontent, as well as the size of the content, to meet end-userpreferences or device constraints such as available storage space. Inaddition, certain portions of these documents may be encrypted, eitherfor confidentiality reasons or for commercial reasons (e.g., pay toread), while other portions of these documents may be in plain text.

In the present embodiment, encoder 32 encodes the input data using anencoding scheme such as an encoding scheme based on JPEG2000. Asmentioned above, encoder 32 can provide other functionality. As a resultof the encoding process, a bit stream such as that exemplified by bitstream 10 of FIG. 1, or a file containing such a bit stream, isgenerated. The file or bit stream is organized according to the encodingscheme; that is, the bits are in a certain order that is established bythe encoding scheme. The output of encoder 32 includes what is referredto herein as scalable data, because the encoded data can be subsequentlyscaled by a transcoder or decoder.

Another output of encoder 32 includes what is referred to herein asscalable profile data. In essence, scalable profile data includes across-reference of scalable attribute values and corresponding datasegments within the scalable, encoded data. For example, the scalableprofile data may be in the form of an index or lookup table thatcross-references data segments and scalable attribute values,exemplified by Table 1 below (with reference also to FIG. 2).

TABLE 1 Exemplary Scalable Profile Data Scalable List of Data SegmentsAttribute (Bit Numbers) Full Bit Rate (B) 0-N  B/2 (R2, Q2) 0-b1 andb2-b3 B/2 (R1, Q3) 0-b2 B/4 0-b1

It is appreciated that the format of Table 1 is exemplary only, and thatthe scalable profile data can be stored in just about anycomputer-readable format. In Table 1, bits are identified by their bitnumbers, but other addressing mechanisms may be used. Also, instead ofidentifying data segments by their bit range, other mechanisms can beused to identify data segments. In general, the scalable profile dataincludes enough information to correlate a value of scalable attributewith one or more data segments in the bit stream 10 of FIG. 1.

Furthermore, Table 1 only addresses bit rate (B), resolution (R) andquality (Q), but in actuality such a table can include any scalableattributes and scalable attribute values selected by the user as inputto encoder 32 (FIG. 3). In addition, the scalable profile data caninclude multiple choices of data segments for each scalable attributevalue. This can provide greater flexibility downstream at the transcoderor decoder. For example, in one application, a bit rate reduction ofone-half can be achieved at a resolution level 2 (R2) and quality level2 (Q2), or at resolution level 1 (R1) and quality level 3 (Q3). R1 maybe better than R2, and Q2 may be better than Q3, so that the same bitrate can be achieved but with tradeoffs in resolution and quality. Thus,by including multiple choices of data segments for each scalableattribute value in the scalable profile data, a user at a downstreamnode (e.g., at a transcoder or decoder) can select the type and degreeof scaling that suits his or her needs, for example.

In one embodiment, the scalable profile data also includes informationabout the distortion that will be produced depending on which segmentsare extracted and hence which are discarded. The distortion can bemeasured in terms of conventional mean-squared error (MSE) or in termsof a perceptual distortion. The scaler (e.g., transcoder) can use theinformation about distortion to determine which segments are mostimportant and should be retained (e.g., extracted), and which segmentshave lesser importance and can be discarded. There may be a separatedistortion parameter per data segment, or per group of segments, or perscalable attribute value (e.g., image resolution, quality level, bitrate, etc.). Accordingly, the decision as to which segments to extractor discard can be performed by accounting for the associated distortionsin combination with the type and degree of scaling that suits theend-user's needs, as well as the type and degree of scaling selectedaccording to network performance characteristics, downstream devicecapabilities, and other factors.

The distortion associated with each data segment, group of segments orscalable attribute can be a measured distortion or an estimateddistortion. For example, the measured distortion associated withdiscarding a particular data segment can be computed by, in essence,dropping that data segment from the bit stream and computing theresulting distortion that would be produced when decoding the remainingdata packets. This measurement can be performed as the media are encodedor for pre-encoded data. For pre-encoded data, a data segment can bedropped from the encoded (compressed) bit stream, the remainder of thedata can be decoded, and the resulting distortion can be computed. As analternative to measured distortion, estimated distortion can beaccomplished, for example, by extracting information from the encodedbit stream that provides an indication of the distortion that may resultshould a particular data segment be discarded. Measured distortion ismore accurate than estimated distortion; however, measured distortion ismore complex to compute.

Note that the predicted distortion can be a single number correspondingto the expected distortion, or it may be a distribution of expecteddistortions, or something in between (e.g., expected distortions with atolerance band corresponding to, for example, one standard deviation).Alternatively, some form of cumulative distribution function for thedistortion can be determined.

In the embodiment of FIG. 4A, the scalable data and the scalable profiledata are stored together. For example, the scalable profile data can beappended to the bit stream or file that contains the scalable data. Forinstances in which there is a large quantity of scalable data, portionsof the scalable profile data may be spaced at intervals within the bitstream or file.

Alternatively, the scalable profile data can be stored and handledseparately from the scalable data, as illustrated by FIG. 4B. In theembodiment of FIG. 4B, it is not even necessary that the scalableprofile data travel with the scalable data. For example, with referenceto FIG. 2, the scalable profile data can travel to transcoder 34,decoder 36 or decoder 38 by a path that is different from the pathtraveled by the scalable data. Alternatively, the scalable profile dataand the scalable data can be stored in separate locations, and thenaccessed and correlated by transcoder 34, decoder 36 or decoder 38whenever necessary.

In operation, the encoder 32 of FIGS. 4A and 4B functions as follows. Inone embodiment, values of scalable attributes are input by a user.Alternatively, values of scalable attributes may be automaticallyselected based on information known by encoder 32 about networkperformance, downstream device capabilities, and the like. Networkperformance characteristics such as available bandwidth may bemonitored, and that information fed to encoder 32. Also, downstreamdevices (including the end-user device) may communicate directly withencoder 32.

Encoder 32 then encodes the input data in a conventional manner, usingthe encoding scheme it is employing (e.g., a JPEG2000 encoding scheme).In addition, the encoder 32 generates scalable profile data for theencoded bit stream (the scalable data). That is, in one embodiment, foreach of the input values of scalable attributes, encoder 32 identifiescorresponding data segments within the encoded data.

Importantly, according to the embodiments of the present invention, thescalable (encoded) data that is output by encoder 32 can be scaledwithout requiring knowledge of the encoding scheme employed by encoder32. Whether the data are encoded using a JPEG2000 scheme or some otherencoding scheme, a transcoder or decoder need only specify the type ofscaling to be performed (e.g., reduce bit rate by one-fourth) toextract/parse from bit stream 10 (FIG. 1) the data segment(s) associatedwith that type of scaling.

In one embodiment, encoder 32 can identify several choices orcombinations of data segments that correspond to a particular value of ascalable attribute. For example, to achieve a bit rate reduction ofone-half (B/2), encoder 32 may identify several different combinationsof data segments that are satisfactory. For image data, one combinationmay result in the reduced bit rate being applied to all portions of theimage product. Another combination may result in full bit rate (B) beingapplied to a region of interest in the image product, and a bit ratereduced by a greater amount (e.g., B/4) being applied to other regionsof the image product, such that the average bit rate is B/2.

At the encoding stage, encoder 32 can apply intelligence (eitherprogrammed intelligence or intelligence based on user input) to decidewhich combination or combinations of data segments to include in thescalable profile data. Alternatively, all combinations can be included;the transcoder or decoder can then decide which combination of datasegments to use based on user input or other considerations such asnetwork performance characteristics or end-user device capabilities.

Note that the data ultimately included by encoder 32 in the encoded bitstream or file can depend on the input values of the scalableattributes. For example, suppose that there is not a need for thescalable data to include data corresponding to the full bit rate case.Instead, only the cases of B/2 and B/4 are to be considered. Encoder 32identifies data segments corresponding to B/2 and B/4, but these datasegments do not encompass all of the encoded data. That is, there issome portion of the encoded data that is not indexed to B/2 or B/4 inthe scalable profile data. If that portion of data is not associatedwith another scalable attribute, encoder 32 can decide to not includethat data in the encoded bit stream or file.

In another embodiment, data segments associated with various values ofscalable attributes may be defined ahead of time. In essence, thescalable profile data exemplified by Table 1 is established in advanceof the data encoding. In this embodiment, the data are encoded and thenplaced in the bit stream in an order that corresponds to the orderdefined by the scalable profile data. For example, referring to FIG. 2,the scalable profile data may define in advance that bits 0-b2 arereserved for bit rate reduction by one-half (B/2). Accordingly, encoder32 will compress the data by the amount necessary to fit data associatedwith B/2 into bits 0-b2, and will place that data into those locationswithin the bit stream 10.

In yet another embodiment, an encoded file, organized according to anencoding scheme, is reorganized based on desired goals and on knowledgeof the scalable attributes, and then, for example, stored or streamed.For instance, using scalable profile data (e.g., a cross-reference ofdata segment and scalable attribute) to locate relevant data segments inthe encoded file, it may be useful to take an encoded file that isorganized first by color component and second by resolution andreorganize the file so that it is instead ordered first by resolutionand second by color component. By taking advantage of the scalableprofile data, the reorganization can be achieved without requiringknowledge of the details of the encoding format.

FIG. 5A shows information flowing into and out of a data scaler 54according to one embodiment of the present invention. Data scaler 54 maybe a transcoder such as transcoder 34 of FIG. 3, or it may be a decoderthat performs a scaling or parsing function (e.g., decoder 38 of FIG.3).

As input, data scaler 54 of FIG. 5A receives scalable data and scalableprofile data. As mentioned above in conjunction with FIGS. 4A and 4B,the scalable data and scalable profile data may be traveling together(e.g., in the same bit stream or file) or apart. The input may bestreamed to data scaler 54 (from encoder 32, for example), or it mayretrieved by data scaler 54 from some type of storage element.

Using the scalable profile data, scaler 54 can parse the scalable dataand create a scaled version of the encoded data. For example, if scaler54 wants to reduce bit rate by one-fourth, it can determine from thescalable profile data which bits in the encoded data can be selected toachieve the desired bit rate reduction (see Table 1, for example).Scaler 54 can then create a scaled version of the encoded data using theidentified data segments, by either discarding the bits not required, orby extracting the identified data segments and combining them into a newbit stream or file.

As mentioned above, the scalable profile data presented to scaler 54 caninclude multiple choices of data segments for each scalable attributevalue (refer to the discussion in conjunction with Table 1). Thedecision as to which segment(s) to extract or discard can be performedby accounting for the type and degree of scaling that suits theend-user's needs, the type and degree of scaling selected according tonetwork performance characteristics and/or downstream devicecapabilities, the distortion information included in the scalableprofile data, or other factors, and combinations thereof. For example,if scaler 54 is to reduce bit rate by one-half, but is presented withdifferent ways of achieving that bit rate reduction, scaler 54 can makea selection considering the characteristics of the end-user deviceand/or the amount of distortion associated with each possible selection.Perhaps the end-user device is a mobile phone with a relatively smalldisplay screen that permits color displays, in which case scaler 54 mayextract data segments that achieve the bit rate reduction at a lowerresolution while preserving color components, or that achieve thedesired bit rate reduction while minimizing the amount of distortion.

Importantly, scaler 54 can perform its functions without requiringknowledge of the encoding scheme that was used to encode the data. Also,scaler 54 can perform these functions without decrypting the encodeddata, if the data are encrypted. To parse the scalable data, scaler 54simply identifies which bits (data segments) to extract, locates thosebits (data segments) in the bit stream or file, and extracts those bits(data segments). Scaler 54 does not need to read the bits in the datasegments, nor does scaler 54 need to read the information in header 11(FIG. 1) to locate the bits (data segments). As such, the scalingoperation can be efficiently accomplished, without unduly taxingcomputational resources. Furthermore, because scaler 54 does not need toread the bits in the data segments, scaler 54 does not require knowledgeof the encoding scheme. Accordingly, scaling can be accomplished onlegacy devices even when the encoding scheme is relatively new andperhaps unknown to scaler 54. Specifically, scaler 54 does not need tobe aware of JPEG2000 in order to scale the encoded data.

Moreover, the data scaling function may be performed by a storage deviceor driver without requiring knowledge of the encoding syntax. Forexample, a storage device (e.g., a disk drive or DVD player) may scalethe encoded data, passing only the appropriate data segments to theactual decoder. In this manner, resources or time are not wasted sendingunnecessary information. This can also simplify the decoder, because thedecoder does not need to perform this processing.

To locate and extract the data segments dictated by the scalable profiledata, scaler 54 can be provided with the capability to read the scalableprofile data by various means. For example, a driver can be preloadedonto scaler 54, or such a driver can be provided with the scalableprofile data. Alternatively, the scalable profile data can be based on,for example, the Extensible Markup Language (XML), which can be read andacted on by the scaler 54.

As a result of the scaling operation, the scalable profile data that wasreceived by scaler 54 may need to be modified. For example, after theencoded data are scaled, some data segments referenced in the scalableprofile data may not be present in the encoded data, or some scalingoperations may no longer be possible. Furthermore, some scalingoperations may need to be redefined in terms of their associated datasegments, because after scaling the data segments may be identified bydifferent bit numbers. There may be other reasons for modifying thescalable profile data.

FIG. 5B shows information flowing into and out of a data scaler 54according to a second embodiment of the present invention, in whichscaler 54 creates modified scalable profile data. Scaler 54 can modifythe scalable profile data in much the same way that encoder 32 createdthe scalable profile data. In the present embodiment, the modifiedscalable profile data is then output from scaler 54, either togetherwith or separate from the scaled version of the encoded data, asdescribed above.

FIG. 6 is a flowchart 60 of a process for scaling (e.g., transcoding)data according to one embodiment of the present invention. FIG. 7 is aflowchart 70 of a process for encoding data according to one embodimentof the present invention. FIG. 8 is a flowchart of a process 80 fordecoding data according to one embodiment of the present invention.Although specific steps are disclosed in flowcharts 60, 70 and 80, suchsteps are exemplary. That is, embodiments of the present invention arewell-suited to performing various other steps or variations of the stepsrecited in flowcharts 60, 70 and 80. It is appreciated that the steps inflowcharts 60, 70 and 80 may be performed in an order different thanpresented, and that not all of the steps in flowcharts 60, 70 and 80 maybe performed. All of, or a portion of, the methods described byflowcharts 60, 70 and 80 may be implemented using computer-readable andcomputer-executable instructions which reside, for example, incomputer-usable media of a computer system.

Generally, flowchart 60 is implemented by scaler 54 of FIGS. 5A and 5B(e.g., transcoder 34 or decoder 38 of FIG. 3); flowchart 70, by encoder32 of FIGS. 3, 4A and 4B; and flowchart 80, by decoder 36 of FIG. 3.However, as mentioned above, different functional blocks can performdifferent functions, one device may perform multiple functions, or afunction or functions may be distributed across multiple devices.

Referring first to FIG. 6, in step 61, a sequence of encoded data isaccessed (e.g., in a bit stream or a file). The encoded data areorganized according to an encoding scheme that was used to encode thedata. In one embodiment, the encoding scheme is based on the JPEG2000standard. Some or all of the encoded data may also be encrypted.

In step 62, a value for a scalable attribute is determined. The scalableattribute identifies how the encoded data are to be scaled. A scalableattribute may be, for example, bit rate (B), and the value of thescalable attribute may be B/4.

In one embodiment, the value for the scalable attribute is received fromanother device in communication with the scaling device. In one suchembodiment, when the other device is to be the recipient of the scaledversion of the encoded data, the encoded data are to be scaled based onthe characteristics of that other device. To determine thosecharacteristics, the scaling device can access profile information thatprovides the characteristics of the other device, and select the valueof the scalable attribute according to the profile information.Alternatively, the scaling device can receive profile information fromthe other device, and select the value of the scalable attributeaccordingly.

In step 63, information (e.g., scalable profile data) that includes areference to segments of the encoded data that are associated with thescalable attribute is accessed. Importantly, the reference is beyond thesyntax of the encoding scheme.

In one embodiment, the reference information (e.g., scalable profiledata) is stored with the sequence of encoded data. In anotherembodiment, the reference information is stored separately from thesequence of encoded data.

In step 64, the scalable profile data is used to locate the segments inthe encoded data. Importantly, the segments are found without requiringknowledge by the scaling device of the encoding scheme.

In step 65, a scaled version of the encoded data is created using thesegments. Also, the scaled version of the encoded data is createdwithout decrypting encrypted data.

In step 66, in one embodiment, the scaled version of the encoded data isforwarded to the downstream device.

In step 67, in one embodiment, modified reference information (e.g.,modified scalable profile data) is generated based on the scaled versionof the encoded data. Segments in the scaled version are associated withselected values of scalable attributes, independent of the encodingscheme, thereby allowing another device to locate the segments in thescaled version and to scale the scaled version without requiringknowledge of the encoding scheme. The modified reference information canbe stored with or separate from the scaled version of the encoded data.

Referring now to FIG. 7, in step 71, data are encoded using an encodingscheme and stored in a file. In one embodiment, the encoding scheme issubstantially compliant with the JPEG2000 standard.

In step 72, locations of data segments in the file are identified.

In step 73, the data segments and selected values of scalable attributesare indexed. A scalable attribute specifies how encoded data are to bescaled in a subsequent scaling operation. The index is independent ofthe encoding scheme, and allows a device to locate the segments and toscale encoded data without requiring knowledge of the encoding scheme.In one embodiment, the selected values of scalable attributes are inputto the encoding device.

In step 74, reference information including the index of the datasegments and the selected values of scalable attributes is stored. Inone embodiment, the index is added to the file of encoded data. Inanother embodiment, the index is stored separately from the file.

In step 75, in one embodiment, at least some of the encoded data isencrypted. In one such embodiment, each data segment of encoded data isprogressively encrypted.

Now with reference to FIG. 8, in step 81, a sequence of encoded data isaccessed. The encoded data are ordered according to an encoding schemeused to encode the data. In one embodiment, the encoding scheme issubstantially compliant with the JPEG2000 standard.

In step 82, the decoding device determines how the encoded data are tobe scaled for decoding.

In step 83, information (e.g., scalable profile data) that identifiescertain segments of the encoded data is accessed. This information is inaddition to the syntax of the encoding scheme. The data segments areidentified based on how the encoded data are to be scaled for decoding.

More specifically, in one embodiment, a value for a scalable attributeis determined. The value may be a user input, or it may be derived basedon the characteristics and capabilities of the decoding device. Thevalue of the scalable attribute identifies how the encoded data are tobe scaled. The scalable profile data includes a reference from thescalable attribute value to certain data segments in the encoded data.

In step 84, using the information from step 83, the data segments arefound in the encoded data. Importantly, the segments are located withoutrequiring knowledge by the decoding device of the encoding scheme.

In step 85, the encoded data included in the data segments found in step84 are decoded. If the encoded data is encrypted, then the data in thedata segments can also be decrypted.

In summary, embodiments of the present invention allow scaling (e.g.,transcoding) of encoded data in a secure and computationally efficientmanner. Scaling can be accomplished without requiring knowledge of thescheme used to encode the data, and accordingly scaling can beaccomplished on legacy devices even when the encoding scheme isrelatively new and perhaps unknown to the scaling device.

Protection Profile Data Generating System and Method

FIG. 9A shows a system for coupling data with a scalable media accordingto one embodiment of the present invention. FIG. 9A illustratesinformation flow into and out of encoder 32 according to one embodimentof the present invention. In the embodiment of FIG. 9A, data isassociated with scalable media that identifies the portions of thescalable media that may combined in order to produce media that isscaled to possess a desired scalable attribute. Thereafter, the scalablemedia is encrypted. In addition, data is associated with the individualportions of the scalable media that identifies the protection attributesof the encryption scheme that is used to encrypt the individual portionsof the scalable media.

In the present embodiment, encoder 32 receives input data (e.g., datasuch as image data that are to be encoded). Encoder 32 also receivesdata that details the attributes of the encryption scheme used toencrypt the input data.

For purposes of the present application, a protection attribute isdefined as an attribute having a range parameters and associated valuesthat specifies how the encoded data are to be protected. Protectionattributes can include but are not limited to encryption primitives,encryption modes, CCSs, and mapping of crypto to scalable mediasegments.

Protection attribute values are values specified for the protectionattributes. For example, protection attribute values for encryptionprimitive can include DES, 3-DES, AEC, etc. Protection attributes andvalues can be similarly defined for other types of data (e.g., audiodata, graphics data, and the like). For example, a protection attributevalue for audio data can indicate whether an audio track is protectedusing MAC or keyed-hashes as crypto-graphic checksums.

As previously discussed, encoder 32 encodes the input data using anencoding scheme such as an encoding scheme based on JPEG2000. Asmentioned above, encoder 32 can provide other functionality. As a resultof the encoding process, a bit stream such as that exemplified by bitstream 10 of FIG. 1, or a file containing such a bit stream, isgenerated. The output of encoder 32 includes what is referred to hereinas scalable data, because the encoded data can be subsequently scaled bya transcoder or decoder.

Referring again to FIG. 9A, another output of encoder 32 includes whatis referred to herein as protection profile data. In essence, protectionprofile data includes a cross-reference of protection attribute valuesand corresponding data segments within the scalable data. For example,the protection profile data may be configured as an index or lookuptable that cross-references data segments and protection attributevalues, as exemplified in Table 2 below (with reference also to FIG. 2).

TABLE 2 Exemplary Protection Profile Data Protection List of DataSegments Attribute/Values (Byte Numbers) Encryption 0-N  Primitive/AESEncryption 0-b1 and b2-b3 Mode/CBC  CCS/MAC 0-b2 Mapping/Multiple 0-b1Keying Digital 0-N  Signature/DSS

It is appreciated that the format of Table 2 is exemplary only, and thatthe protection profile data can be stored in just about anycomputer-readable format. In Table 2, bytes are identified by their bytenumbers, but other addressing mechanisms may be used. Also, instead ofidentifying data segments by their byte or bit or block (e.g. 8 byte)range, other mechanisms can be used to identify data segments. Ingeneral, the protection profile data includes enough information tocorrelate a value of a protection attribute with one or more datasegments in the bit stream 10 of FIG. 1.

It should be appreciated that Table 2 can include any protectionattributes and protection attribute values selected by the user as aninput to encoder 32 (FIG. 3). In addition, the list of data segments towhich a particular protection attribute is to be applied can includemultiple choices of data segments for each protection attribute value.This can provide a wide range of data protection flexibility at theencoder. Thus, the choice flexibility of exemplary embodiments, thatallows the inclusion of multiple choices of data segments for eachprotection attribute value in the protection profile data, allows a user(e.g., providing input to the encoder) to select the type and degree ofprotection that suits his or her needs.

It should be noted that the scalable data (encoded and encrypted) thatis output by encoder 32 can be scaled without requiring knowledge of theencoding scheme, or the encryption scheme employed by encoder 32.Whether the data are encoded using a JPEG2000 scheme or some otherencoding scheme, a transcoder or decoder need only specify the type ofscaling to be performed (e.g., reduce bit rate by one-fourth) toextract/parse from bit stream 10 (FIG. 1) the data segment(s) associatedwith that type of scaling.

In the embodiment of FIG. 9A, the scalable data and the protectionprofile data are stored together. For example, the protection profiledata can be appended to the bit stream or file that contains thescalable data. For instances in which there is a large quantity ofscalable data, portions of the scalable profile data may be spaced atintervals within the bit stream or file.

Alternatively, the protection profile data can be stored and handledseparately from the scalable data, as illustrated by FIG. 9B. In theembodiment of FIG. 9B, it is not even necessary that the scalableprofile data travel with the scalable data. For example, with referenceto FIG. 3, the protection profile data can travel to transcoder 34,decoder 36 or decoder 38 by a path that is different from the pathtraveled by the scalable data. Alternatively, the protection profiledata and the scalable data can be stored in separate locations, and thenaccessed and correlated by transcoder 34, decoder 36 or decoder 38whenever necessary.

In operation, the encoder 32 of FIGS. 9A and 9B functions as follows. Inone embodiment, values of protection attributes are input by a user,which may include the content creator, content distributor, or contentconsumer. Alternatively, values of protection attributes may beautomatically selected based on information known by encoder 32 aboutnetwork performance, downstream device capabilities, digital rightmanagement (DRM) policies, security concerns or vulnerabilities, and thelike. Network performance characteristics such as available bandwidthmay be monitored, and that information fed to encoder 32. Also,downstream devices (including the end-user device) may communicatedirectly with encoder 32.

Encoder 32 then encodes the input data in a conventional manner, usingthe encoding scheme it is employing (e.g., a JPEG2000 encoding scheme).In addition, the encoder 32 generates protection profile data for theencoded bit stream (the scalable data). That is, in one embodiment, foreach of the input values of protection attributes, encoder 32 identifiescorresponding data segments within the encoded data to which theprotection attribute may be associated.

In one embodiment, encoder 32 can identify several choices orcombinations of data segments that are associated with a particularvalue of a protection attribute. For example, in the image productcontext, for a particular encryption primitive, encoder 32 may identifyseveral different combinations of data segments that may be encryptedusing that encryption primitive. According to one embodiment, onecombination may result in the same encryption primitive being applied toall portions of the image product. Another combination may result in oneencryption primitive being applied to one region of interest in theimage product, and another encryption primitive being applied to otherregions of interest in the image product.

At the encoding stage, encoder 32 can apply intelligence (eitherprogrammed intelligence or intelligence based on user input) to decidewhich combination or combinations of data segments to associate with theprotection profile data. Alternatively, all combinations can beassociated with protection profile data; the encoder can decide whichcombination (e.g., some or all) of data segments to associate with aparticular protection attribute based on user input or otherconsiderations such as network performance characteristics, end-userdevice capabilities, digital rights management policies (DRM), orsecurity vulnerabilities.

In another embodiment, data segments associated with various values ofprotection attributes may be defined ahead of time. In essence, theprotection profile data exemplified by Table 1 is established in advanceof data encoding. In this embodiment, the data are encoded and thenencrypted in a manner that corresponds to the manner defined by theprotection profile data. For example, referring to FIG. 2, theprotection profile data may define in advance that bytes 0-b2 are to beencrypted using an encryption primitive having protection attributevalue AES. The encoder would then encrypt these bytes accordingly.

FIG. 10 shows information flowing into and out of a data scaler 54according to a second embodiment of the present invention, in whichscaler 54 creates modified protection profile data. Scaler 54 can modifythe protection profile data in much the same way that encoder 32 createdthe protection profile data. In the present embodiment, the modifiedscalable profile data is then output from scaler 54, either togetherwith or separate from the scaled version of the encoded data, asdescribed above.

In the present embodiment, protection profile data that is associatedwith individual portions of the scalable media identifies protectionattributes that can include but is not limited to encryption primitives,encryption modes, cryptographic checksums, and the mapping of crypto toscalable media such as is shown in Table 2. Moreover, the protectionprofile data is used in the decryption of an encryption algorithm and acryptographic technique that is used in the encryption scheme used toencrypt the individual portions of the scalable media.

In the present embodiment, respective individual portions of thescalable media may have different keys that are used to decrypt theindividual portions of the scalable media, may use different encryptionalgorithms, and may use different CCSs. It should be appreciated that,respective classes of keys may be required to decrypt respective classesof scalable media. For example, a first key may be required to gainaccess to a file containing media having a first resolution while asecond key may be required to gain access to a file containing mediahaving a second resolution.

According to one embodiment, parameters of the protection profile datamay change over time. It should be appreciated that the protectionprofile data can be remapped to reflect changes made to a sequence ofdata to which it is associated.

FIG. 11 shows the steps performed in a method for scaling portions of ascalable media according to one embodiment of the present invention. Atstep 1101, data is associated with scalable media that identifiesportions of the scalable media to combine in order to produce media thatis scaled to possess a desired scalable attribute. In the presentembodiment, the scalable profile data that is associated with theindividual portions of scalable media may be coupled to the scalablemedia or signaled remotely. At step 1103, the portions of the scalablemedia identified in step 1101, are encrypted. According to oneembodiment, as discussed herein, the scalable media portions thatconstitute the scalable media can be encrypted using the same ordifferent schemes.

At step 1105, data is associated with the identified portions of thescalable media that identifies protection attributes of the encryptionscheme used to encrypt the identified portions of the scalable media. Inthe present embodiment, the protection profile data that is associatedwith the individual portions of scalable media may be coupled to thescalable media or signaled remotely.

FIG. 12 is a flowchart of the steps performed in a method of scalingencrypted scalable media without decoding according to one embodiment ofthe present invention. At step 1201, data is accessed that is associatedwith the encrypted scalable media and that identifies portions of theencrypted scalable media to combine in order to produce media that isscaled to possess a desired scalable attribute. At step 1203, one ormore of the portions of the encrypted scalable media and associatedcryptographic checksums are combined into a transmittable sequence ofdata. At step 1205, data associated with the scalable media thatidentifies portions of the scalable media to combine in order to producemedia that is scaled to possess a desired scalable attribute isremapped.

At step 1207, data associated with the portions of the scalable mediathat identifies protection attributes of the encryption system used toencrypt portions of the scalable media is remapped. It should beappreciated that one or more portions of the encrypted scalable mediaand associated cryptographic checksums can constitute a scaled versionof the encrypted scalable media.

FIG. 13 is a flowchart of the steps performed in a method of decodingscalable media according to one embodiment of the present invention. Atstep 1301, data associated with portions of the scalable media thatidentifies protection attributes of the encryption system used toencrypt the portions of the scalable media are accessed. At step 1303,data associated with the scalable media that identifies portions of thescalable media to combine in order to produce media that is scaled topossess a desired scalable attribute is accessed.

At step 1305, the portions of the scalable media are decrypted based onthe data associated with the portions of the scalable media thatidentifies protection attributes of the encryption system used toencrypt them. At step 1307, the portions of the scalable media aredecoded based on the data associated with the scalable media thatidentifies portions of the scalable media to combine in order to producemedia that is scaled to possess a desired scalable attribute.

In summary, methods for associating data with portions of a scalablemedia are disclosed. Data is associated with the scalable media thatidentifies portions of the scalable media to combine in order to producemedia that is scaled to possess a desired scalable attribute. Portionsof the scalable media are encrypted. Data is associated with theportions of the scalable media that identifies protection attributes ofthe encryption scheme used to encrypt the portions of the scalablemedia.

Scaling Progressively Encrypted Data Without Knowledge of EncodingScheme

FIG. 14 illustrates a method for encoding a progressively encryptedsequence of scalable data according to one embodiment of the presentinvention. In the FIG. 14 embodiment, data (e.g., scalable profile datadiscussed with reference to FIGS. 4A and 4B) is associated with asequence of scalable data that identifies combinable portions of thesequence of scalable data to combine in order to produce a scaledversion of the progressively encrypted sequence of data. In the presentembodiment, the scaled version of the progressively encrypted sequenceof scalable data is scaled without knowledge of the encoding scheme ofthe progressively encrypted sequence of scalable data. Moreover, acryptographic checksum (CCS) can be computed and associated with atleast one combinable portion of the progressively encrypted sequence ofscalable data. FIG. 14 shows a progressively encrypted sequence of mediacontent 301 that is separated, at 312, into segments 304, 305, 306 and307 (e.g., combinable portions).

It should be appreciated that each segment (e.g., combinable portion)can include at least one independently decodable part. In the FIG. 14embodiment, the independently decodable parts are labeled A, B, C and Dwith each segment including only one independently decodable part. Inalternate embodiments, more than one independently decodable part can beincluded in a segment.

According to one embodiment, a cryptographic checksum can be computedfor each segment 304, 305, 306 and 307. In the FIG. 14 embodiment, at313, after the computation of cryptographic checksums, each identifiedsegment 304, 305, 306 and 307 is associated with a correspondingcryptographic checksum 314, 315, 316 and 317. The identified segments(e.g., combinable portions) and their associated cryptographic checksumsare combined into a media segment 321, 322, 323 and 324.

It should be appreciated that the length of each segment, in embodimentsof the present invention, is chosen so that its length plus that of itsassociated CCS is less than the maximum media segment payload sizeallowable or the maximum transmittable unit (MTU) allowable for thenetwork. As discussed herein, a media segment can comprise a singletruncatable unit or a plurality of truncatable units. It is noted alsothat, though there are four media segments illustrated in FIG. 14, anynumber of media segments can be used.

It should be appreciated that the terms “independently decodable part”and “Iruncatable unit” can have different meanings in the descriptionsof embodiments of the present invention made herein. An independentlydecodable part of a media segment is a portion of the media segment'spayload that can be decoded without the necessity of decoding otherportions of the media segment payload. Moreover, if encrypted, theindependently decodable part can be decrypted without the need todecrypt the remainder of the payload. A truncatable unit is a portion ofa media segment payload that can be truncated from the media segment,with or without decryption, without detrimentally effecting theremainder of the media segment. It should be appreciated that althoughthe terms can be used somewhat interchangeably, in the discussions madeherein they maintain their separate meanings.

FIG. 15A illustrates the transcoding by truncation of a progressivelyencrypted sequence of scalable media content according to one embodimentof the present invention. In the embodiment illustrated in FIG. 15A,media content 411 is separated, at 412, into segments, that eachcomprise one or more independently decodable parts (A, B, etc.). Asdiscussed above with reference to FIG. 14, a media segment's payload cancomprise one or more independently decodable parts. Moreover, accordingto some embodiments, the independently decodable parts can furthercomprise independently truncatable units.

In the FIG. 15A embodiment, a cryptographic checksum is computed foreach of the decodable parts or truncatable units of media segment 413 inorder of priority and for the entire preceding packet payload. First, acryptographic checksum 415 is computed for the first truncatable unit A(e.g., 403) resulting in CCS(A) 415. Subsequently, a cryptographicchecksum 416 is computed for the entire preceding media segment payloadwhich includes independently decodable part A (e.g., 403), cryptographicchecksum CCS(A) (e.g., 415), and independently decodable part B (e.g.,404). The resulting checksum is shown in FIG. 4A as CCS(A,CCS(A),B),416. It should be appreciated that, if a third independently decodablepart were included, the next cryptographic checksum could be representedas:CCS(A,CCS(A),B,CCS(A,CCS(A),B),C).

In the present embodiment, a transcoder-readable header, 414, can beassociated with media segment 413. According to exemplary embodiments,the transcoder readable header can contain information such as thelocation of the truncation points in scalable media segments, such asmedia segment 413, and scalable profile data that identifies segments ofthe scalable media segment data (e.g., media segment) that can beextracted in order to produce media that is scaled to possess a desiredscalable attribute. In alternate embodiments, scalable profile data canreside in the middle the end or at different locations in the scalablemedia segment.

In the present embodiment, when a transcoding session is conducted(e.g., 417) the transcoder can truncate selected truncatable units inorder to achieve a desired scaling result. According to exemplaryembodiments, if a media segment being transcoded is encrypted,decryption is not required in order to perform this form of transcoding.In the example illustrated in FIG. 14A, truncatable unit B and itsassociated cryptographic checksum, 416, are truncated. The untruncated,and undecrypted, remainder of the media segment, 418, is then forwardedwith its necessary cryptographic checksum, CCS(A) 415, independentlydecodable part A 403, and header 414, intact.

In the embodiment of the present invention illustrated in FIG. 15A, anynumber of truncatable units and their associated cryptographic checksumcan be truncated from a media segment as necessary to meet transcodingrequirements (e.g., to obtain a desired scalable attribute). In eachcase, the truncatable unit, its associated cryptographic checksum andsubsequent truncatable units whose cryptographic checksums includecalculation for the truncated units are also truncated.

FIG. 15B illustrates the transcoding by truncation of a progressivelyencrypted sequence of scalable media content according to an alternateembodiment of the present invention. In the FIG. 15B embodiment, mediacontent 421 is separated at 422 into segments which, as discussed above,may comprise any number of independently decodable parts, shown as A, B,C, D, etc. It is noted that, in this example, some segments comprisemore independently decodable parts than others because of the size ofeach part (see FIG. 15B). In the FIG. 15B embodiment, the selection ofthe number of independently decodable parts is, predicated on themaximum size of the media segment. For example, independently decodableparts A, (e.g., 433), B (e.g., 434), and C (e.g., 435), are combined inone segment while independently decodable part D, 436, constitutes asimilarly-sized segment by itself.

In the embodiment illustrated in FIG. 15B, a cryptographic checksum iscalculated for each independently decodable part. According to thisembodiment, the calculation of the cryptographic checksum is madeindependently of other independently decodable parts. Media segment 423is formed from the combination of independently decodable part A (e.g.,433) and CCS(A) (e.g., 425), independently decodable part B (e.g., 434)and CCS(B) (e.g., 426), and independently decodable part C (e.g., 435)and CCS(C) (e.g., 427).

In the embodiment illustrated in FIG. 15B, the transcoding 428, of mediasegment 423 then involves truncating the selected independentlydecodable parts, or truncatable units, and their associatedcryptographic checksums. In the example illustrated in FIG. 15B,truncatable unit C (e.g., 435) and CCS(C) (e.g., 427) are truncated. Inthis embodiment, transcoder-readable header 424 remains intact andretains it's information regarding the truncation points. Truncatedmedia segment 429, therefore, is available to be transcoded bytruncation subsequently by truncating either truncatable unit B (e.g.,434) and CCS(B) (e.g., 426) or truncatable unit A (e.g., 433) CCS(A)(e.g., 425).

In this fashion, transcoding can occur at any desired point in a networkwithout any media segment having to be decrypted and re-encrypted toachieve a desired transcoding result. After each truncation, theremainder of the media segment, undecrypted, retains its necessarycryptographic checksums, and the security of the data remains intact.

It should be appreciated, in some embodiments, transcoding can beexecuted by deleting entire media segments from the media content. It isalso noted that cryptographic checksums can also be used in unencryptedmedia streams, such as for media segment verification.

FIG. 16A illustrates the transcoding of a progressively encryptedsequence of scalable media content according to one embodiment of thepresent invention. In the FIG. 16A embodiment, media content 500(constituted by scalable data portions A, B and C, etc.) is separated(e.g., 501), into truncatable units (e.g., 502). Each media segmentpayload, in the secure media stream, is encrypted (e.g., 503) andappended with an independent cryptographic checksum (CCS). Referring toFIG. 16A, the cryptographic checksums are designated as CCS(A) (e.g.,506), CCS(B) (e.g., 507), and CCS(C) (e.g., 508). According to exemplaryembodiments, the encrypted truncatable units, along with theirassociated cryptographic checksums, are combined to form appropriatelength transmittable media segment (see FIG. 16A).

It is noted here that cryptographic checksums may be of many differenttypes. A common checksum can involve a well-known hash function, whichprovides a fingerprint of the data contained in an encrypted mediasegment and can guarantee the authenticity of received data and thevalidity of decrypted data. Other examples of checksum functions thatcan be used to provide cryptographic checksum capability include MessageAuthentication Codes (MAC), keyed hashes such as MD4 & MD5 (MessageDigest algorithms), SHA (Secure Hash Algorithm), RIPEMD (RACE IntegrityPrimitives Evaluation Message Digest), and HMAC (keyed-Hashing forMessage Authentication). Also, in some implementations, digitalsignature schemes may also be used.

In another embodiment of the present invention, the separation of datasegments into truncatable units is referred to as secure scalablestreaming (SSS). Each media segment can be transcoded by truncating themedia segment at appropriate truncation points which may be defined in aheader included in the media segment. For example, during transcoding,bit rate reduction, frame rate reduction, or the like is achieved bytruncating, or eliminating, one or more truncatable units from the mediasegment.

In the FIG. 16A embodiment, a transcoder-readable header 505 is writtenand coupled to a transmittable media segment. As discussed withreference to FIG. 16A, above, the transcoder-readable header (e.g.,scalable profile data, protection profile data, etc.) includesinformation relating to the media segment payloads accompanying themedia segment but does not disclose the contents of the media segment'spayloads. By reading the transcoder-readable header, the transcoder candelete portions (e.g., 509) or “scale down” a transmittable mediasegment without decrypting either the deleted part or the remainder ofthe media segment, as is illustrated in FIG. 16A at 510. In the exampleshown, independently decodable part B with its associated cryptographicchecksum, is deleted with no effect on independently decodable parts Aor C or their associated cryptographic checksums. Consequently,end-to-end security of the streamed media data is maintained and areceiver of the streamed media can validate the integrity of thetranscoded data.

In the embodiments illustrated in FIGS. 16A and 16B, after transcodingis completed, a new transcoder-readable header, 511, may be written toreflect the content of the newly constituted media segment, 510. As inthe previous transcoder-readable header, information about the start andend points of the included media segment payloads and the media segmentpayload priority can be included but information disclosing the contentsis not. Moreover, because the new transcoder-readable header can bewritten by a transcoder that does not have the key with which to decryptthe media segment payloads or to evaluate the CCSs, the newtranscoder-readable header is not capable of disclosing media segmentpayload contents.

With a new transcoder-readable header, possible further transcoding andscaling can be performed at other downstream locations. In addition tomedia segment payload size and location, media segment payload prioritycan be included in the transcoder-readable header. For example, priorityinformation can be included, such as that identifying components of aweb page that are considered discardable by the web page owner. In thecase of transmission to a handheld device with a lower displaycapability than a large desktop computer, much of the information incomplex web pages can be lost. By making lower priority informationremovable in early transcoding, valuable bandwidth can be preserved forother uses when transmitting data to such smaller devices.

Embodiments of the present invention are enabled to process mediasegments that are not streamed. FIG. 16B illustrates, in block format,the process of another embodiment. Here, stored data is manipulatedwithout disturbing the cryptographic checksums. Large media segment 514,comprising segments A, B, & C, is taken from storage medium 521. In theexample of FIG. 16B, data is in essence compressed to reduce storagespace. This is accomplished by the removal of a segment of data, in thiscase segment B, and its associated CCS, 517, by transcoding, 519. Theresult of such transcoding is the smaller media segment 522. It is notedthat transcoding in this manner leaves segments A and C intact and,significantly, CCS(A) 516 and CCS(C) 518, are undisturbed. If necessaryto later operations, transcoder-readable header 515 can be replaced bynew transcoder readable header 511.

It is noted that the transcoding schemes provided by embodiments of thepresent invention are not limited to streamed data but can also be usedwith stored data. Additionally, the transcoding techniques are usefulfor unencrypted data as well as encrypted data. In either case,transcoding can occur without disruption of the cryptographic checksumsand without having to read the encoded data.

In the present embodiment, media segments may be any appropriatedivision that allows one or more of the segments and their associatedcryptographic checksums, to fit into a communication media segment. Anexample of appropriately separated independently decodable parts isparts of highly detailed compressed images, such as can be transmittedwith the compression standard developed by the Joint Picture ExpertGroup (JPEG), for example JPEG-2000.

It should be appreciated that in many instances, the first datatransmitted contains data sufficient to produce a highly pixilatedimage. Subsequently transmitted data then successively refines the imagedetail. The presentation of such an image on a large display can makeuse of the enhanced detail. However, the display on a handheld computermay show no difference between the image after the first refinement andthe image after the last and most highly detailed refinement.Consequently, a logical transcode can remove the more highly detaileddata from the stream if the receiver is unable to use the detail. Insome instances of transmitted data, a single packet may contain datacomprising several levels of detail. However, some larger images mayrequire several media segments to carry all the requisite data.

FIGS. 17-19 are flowcharts of the steps performed in methods for scalingprogressively encrypted data without knowledge of its encoding scheme.Although specific steps are disclosed in the flowcharts, such steps areexemplary. That is, embodiments of the present invention are well-suitedto performing various other steps or variations of the steps recited inthe flowcharts. It is appreciated that the steps in the flowcharts maybe performed in an order different than presented, and that not all ofthe steps in the flowcharts may be performed. All of, or a portion of,the methods described by the flowcharts may be implemented usingcomputer-readable and computer-executable instructions which reside, forexample, in computer-usable media of a computer system.

FIG. 17 is a flowchart of the steps performed in a method for encoding aprogressively encrypted sequence of scalable data according to oneembodiment. At step 1701, data is associated with a progressivelyencrypted sequence of scalable data that identifies combinable portionsof the progressively encrypted sequence of scalable data to combine inorder to produce a version of the progressively encrypted sequence ofscalable data that is scaled to possess a desired scalable attribute.According to one embodiment, the scaled version of said progressivelyencrypted sequence of scalable data is scaled without being decoded. Atstep 1703, a cryptographic checksum is computed for at least one portionof the progressively encrypted sequence of scalable data. At step 1705,a cryptographic checksum is associated with at least one portion of theprogressively encrypted sequence of scalable data.

FIG. 18 is a flowchart of the steps performed in a method of transcodinga progressively encrypted sequence of scalable data according to oneembodiment. At step 1801, data associated with the progressivelyencrypted sequence of scalable data is accessed that identifiescombinable portions of the progressively encrypted sequence of scalabledata to combine in order to produce a scaled version of the sequence ofdata that is scaled to possess a desired scalable attribute. Accordingto one embodiment, the scaled version of said progressively encryptedsequence of scalable data is scaled without being decoded. At step 1803,one or more of the combinable portions of the progressively encryptedsequence of scalable data and their associated cryptographic checksumsare combined into a transmittable sequence of data.

FIG. 19 is a flowchart of the steps performed in a method of decoding aprogressively encrypted sequence of scalable data according to oneembodiment of the present invention. At step 1801, an encoded andprogressively encrypted sequence of scalable data is accessed. At step1803, data associated with the encoded and progressively encryptedsequence of scalable data is accessed that identifies how combinableportions of the progressively encrypted sequence of scalable data are tobe scaled to achieve a desired scalable attribute. In the presentembodiment, scaling is accomplished without decoding the progressivelyencrypted sequence of scalable data.

At step 1905, a cryptographic checksum is used in order to authenticatethe contents of at least one combinable portion of the progressivelyencrypted sequence of scalable data. At step 1907, the encoded andprogressively encrypted sequence of scalable data is decoded based onthe data that is associated with the encoded and progressively encryptedsequence of scalable data. This data identifies how combinable portionsof the encoded and progressively encrypted scalable data are to bescaled to achieve a desired scalable attribute.

According to one embodiment, at least one combinable portion of theprogressively encrypted scalable data and a cryptographic checksum canbe encrypted. In exemplary embodiments, at least one combinable portioncan be enabled to be decrypted independently of other combinableportions that constitute the progressively encrypted scalable data.Moreover, a cryptographic checksum can be computed for each combinableportion of the progressively encrypted sequence of scalable data.

According to one embodiment, the at least one combinable portion of theprogressively encrypted sequence of scalable data is enabled to bedecrypted independently of other portions comprising said progressivelyencrypted sequence of scalable data. In the present embodiment, theprogressively encrypted sequence of scalable data comprises a pluralityof combinable portions. Moreover, a first cryptographic checksum can becalculated for a first combinable portion of said progressivelyencrypted sequence of scalable data, and a second cryptographic checksumcan be calculated for the combination of a second combinable portion ofthe progressively encrypted sequence of scalable data, the firstcombinable portion of the progressively encrypted sequence of scalabledata, and the first cryptographic checksum.

According to one embodiment, the cryptographic checksum can be computedusing a hash function. In addition, the data associated with the encodedand progressively encrypted sequence of scalable data can includeinformation related to the combinable portions of the progressivelyencrypted sequence of scalable data and the cryptographic checksums. Itshould be appreciated that the data associated with the encoded andprogressively encrypted sequence of scalable data can enable thetranscoding of the progressively encrypted sequence of scalable data.

According to one embodiment the combinable portions of the progressivelyencrypted sequence of scalable data and the cryptographic checksums canbe enabled to be encrypted independently of a transcoder readableheader. Moreover, the combinable portions of the progressively encryptedsequence of scalable data and the cryptographic checksums can be enabledto be decrypted independently of the data associated with theprogressively encrypted sequence of scalable data. In addition, the dataassociated with the progressively encrypted sequence of scalable datacan be enabled to be read independently of the combinable portions ofthe progressively encrypted sequence of scalable data and thecryptographic checksums.

According to one embodiment, a cryptographic checksum can be computedbased on one of the combinable portions of the progressively encryptedsequence of scalable data. The cryptographic checksum can be computedbased on a plurality of combinable portions and associated checksums.Moreover, the cryptographic checksum can be calculated using a hashfunction. In an alternative embodiment, the cryptographic checksum canbe calculated using a message digest function.

According to one embodiment, the cryptographic checksum can becalculated using a message authentication code function. In anotherembodiment, the cryptographic checksum can be calculated using akeyed-hashing-for-message-authentication function. In yet anotherembodiment, the cryptographic checksum can be calculated using a digitalsignature function.

According to one embodiment, the data associated with the progressivelyencrypted sequence of scalable data is enabled to be writtenindependently of combinable portions of the progressively encryptedsequence of scalable data and the cryptographic checksums. It should beappreciated that each of the combinable portions of the progressivelyencrypted sequence of scalable data can be enabled to be extracted froma transmittable packet independently of other combinable portions of theprogressively encrypted sequence of scalable data in the packet.

In the present embodiment, data that is associated with theprogressively encrypted sequence of scalable data includes informationrelated to the combinable portions of the progressively encryptedsequence of scalable data and a cryptographic checksum. According to oneembodiment, the data associated with the progressively encryptedsequence of scalable data enables the transcoding of a data packet.

It should be appreciated that the combinable portions of theprogressively encrypted sequence of scalable data and the cryptographicchecksums can be enabled to be encrypted independently of a transcoderreadable header. In one embodiment, the combinable portions of theprogressively encrypted sequence of scalable data and the cryptographicchecksums are enabled to be decrypted independently of the dataassociated with the encoded and progressively encrypted sequence ofscalable data. Moreover, the data associated with the progressivelyencrypted sequence of scalable data can be enabled to be readindependently of the combinable portions of the progressively encryptedsequence of scalable data and the cryptographic checksums.

In summary, the methods of the present invention provide a method forscaling a progressively encrypting sequence of scalable data for scalinga progressively encrypted sequence of scalable data without havingknowledge of its encryption scheme. The method includes associating datawith the progressively encrypted sequence of scalable data thatidentifies combinable portions of the progressively encrypted sequenceof scalable data to combine in order to produce a scaled version of theprogressively encrypted sequence of scalable data. The scaled version ofthe progressively encrypted sequence of scalable data is scaled topossess a desired scalable attribute. Moreover, the scaled version ofthe progressively encrypted sequence of scalable data is scaled withoutbeing decoded. A cryptographic checksum is computed for at least onecombinable portion of the progressively encrypted sequence of scalabledata and, a cryptographic checksum is associated with the at least onecombinable portion of the progressively encrypted sequence of scalabledata.

Encoders

FIG. 20 is a block diagram that depicts a scalable media encoder,according to one embodiment of the present invention.

According to one embodiment, scalable media encoder 2020 is any device,such as a portable communications device, a stationary computing device,or a digital image capturing device, that can encode any type ofscalable media, such as JPEG 2000 or MPEG. Examples of portablecommunications devices include, but are not limited to, pocket personalcomputers, laptops, digital cameras, video cameras and cell phones.Examples of stationary computing devices include, but are not limitedto, personal computers, televisions, and servers. The servers may beASIC servers. Examples of digital image capturing devices include, butare not limited to, digital cameras and video cameras.

According to one embodiment, scalable media encoder 2020 receives mediadata 2012, scalable attribute criteria 2014, and protection attributecriteria 2016. The scalable attribute criteria 2014 may list scalableattributes that the scalable media encoder 2020 may use for generatingthe segments of the scalable media 2052 that may be extracted by anotherdevice, as described herein. Examples of scalable attributes include butare not limited to resolution, bitrate, color, as described herein.Examples of devices that may extract segments include but are notlimited to translators and decoders.

In yet another embodiment, as depicted in FIG. 20, scalable mediaencoder 2020 includes a media data receiver 2022 that is configured toreceive media data 2012. Media data 2012 may be scalable media such asan image in JPEG 200 format or a non-scaled image such as a scannedimage of a picture, according to one embodiment.

Scalable media encoder 2020, further includes, as depicted in FIG. 20, ascalable media generator 2024 that is coupled to media data receiver2022, according to one embodiment. For example, in one embodiment,scalable media generator 2024 may generate scalable media 2052, in thecase where media data 2012 is a non-scaled image.

Still referring to FIG. 20, scalable media encoder 2020, also includes ascalable media outputter 2026 that is coupled to scalable mediagenerator 2024 and that is adapted to outputting scalable media 2052, inone embodiment.

Scalable media encoder 2020, as depicted in FIG. 20, further includes ascalable attribute criteria receiver 2032 that is configured to receivescalable attribute criteria 2014, according to one embodiment. In yetanother embodiment, scalable attribute criteria receiver 2032 is coupledto a scalable profile data generator 2034 that is adapted to generatescalable profile data 2054, which may include segments that anotherdevice, such as a translator or a decoder, may extract from scalablemedia 2052 to achieve the desired results for one or more scalableattributes, as described herein. The scalable profile data generator2034, in yet another embodiment, is coupled to a scalable profile dataoutputter 2036 which is adapted for outputting scalable profile data2054.

Still referring to FIG. 20, scalable media encoder 2020, according toone embodiment, further includes a protection attribute criteriareceiver 2042 that is configured to receive protection attributecriteria 2016, as already described herein. The protection attributecriteria receiver 2042, according to still another embodiment, iscoupled to a protection profile data generator 2044 that is adapted togenerate protection profile data 2056 based, at least in part, on theprotection attribute criteria 2016 that scalable media encoder 2020received, according to a embodiments described herein. The protectionprofile data generator 2044 is coupled to a protection profile dataoutputter 2046, which is adapted to output the protection profile data2056, according to another embodiment. In still another embodiment, thescalable media 2052 may be encrypted using the protection profile data2056, according to embodiments already described herein. In yet anotherembodiment, the scalable media 2052 may be unencrypted. There are anumber of protection mechanisms that may be applied. For example,cryptographic checksums may be applied for integrity checking, ordigital signatures for authentication, etc.

According to one embodiment, the scalable attribute criteria receiver2032 and/or the protection attribute criteria receiver 2042 may begeneralized user interfaces (GUIs). For example, a user may use a GUI toenter attributes, such as resolutions, bitrates, etc., to indicate howthe scalable media generator 2024 may generate scalable profile data2054 for those attributes, according to embodiments already describedherein. Similarly, a user may use a GUI to enter attributes, such asData Encryption Standard (DES), encryption modes such as Electronic CodeBook (ECB), etc., to indicate how the protection profile data generator2044 may generate protection profile data 2056 indicating how portionsof scalable media 2052 should be or are protected with those attributes,according to embodiments already described herein.

According to one embodiment, the processing performed by scalable mediaencoder 2020 may be complex or simple. For example, the scalable mediagenerator 2024 of scalable media encoder 2020 may generate scalablemedia 2052 that may be scaled in many alternative ways or with only afew alternative, as already described herein.

Media data 2012, scalable attribute criteria 2014, and/or protectionattribute criteria 2016 may be communicated to scalable media encoder2020 using a network, according to one embodiment. In anotherembodiment, media data 2012, scalable attribute criteria 2014, and/orprotection attribute criteria 2016 may be retrieved by scalable mediaencoder 2020 from a storage device, such as a compact disk (CD), adigital video disk (DVD), or a direct access storage device (DASD).

As depicted in FIG. 20, scalable media 2052, scalable profile data 2054,and/or protection profile data 2056 may be transmitted to anotherdevice, such as a transcoder or a decoder, using a network, according toone embodiment. According to another embodiment, scalable media 2052,scalable profile data 2054, and/or protection profile data 2056 may betransmitted to another device, such as a transcoder or a decoder, via astorage device, such as a compact disk (CD), a digital video disk (DVD),or a direct access storage device (DASD).

In still another embodiment, two or more of the outputters (2026, 2036,2046) may be combined. For example, scalable media outputter 2052 andthe scalable profile data outputter 2036 may be combined into oneoutputter. Similarly, scalable media outputter 2026 and protectionprofile data outputter 2046 may be combined into one outputter.Alternatively, all three outputters (2026, 2036, 2046) may be combinedinto one outputter.

Scalable media 2052, scalable profile data 2054, and protection profiledata 2056 may be separate files or combined together in a single file inany combination, according to one embodiment, For example, scalablemedia 2052, scalable profile data 2054, and protection profile data 2056may be in a single file together. In a second example, scalable media2052 and scalable profile data 2054 may be in a file together, whileprotection profile data 2056 is in a separate file. In a third example,scalable media 2052 and protection profile data 2056 may be in a filetogether, while scalable profile data 2054 is in a separate file.

FIG. 21 is a block diagram that depicts a scalable media encoder,according to another embodiment of the present invention. FIG. 21depicts a scalable media encoder 2120 that is similar to the scalablemedia encoder 2020 depicted in FIG. 20, except that scalable mediaencoder 2120 does not include a protection attribute criteria receiver2042 which receives protection attribute criteria 2016, a protectionprofile data generator 2044 and a protection profile data outputter 2046that outputs a protection profile data 2056.

In one embodiment, the process of encrypting the scalable media 2052 maybe performed using a web page. For example, since scalable media encoder2120 does not include a protection attribute criteria receiver 2042,protection profile data generator 2044, and a protection profile dataoutputter 2046, the scalable media 2052 may be encrypted usingprotection profile data 2056 some where other than at scalable mediaencoder 2020, such as a web page.

FIG. 22 is a block diagram that depicts a scalable media encoder,according to another embodiment of the present invention. FIG. 22depicts a scalable media encoder 2220 that is similar to the scalablemedia encoder 2020 depicted in FIG. 20, except that scalable mediaencoder 2220 does not include a scalable attribute criteria receiver2032 which receives scalable attribute criteria 2014, a scalable profiledata generator 2034 and a scalable profile data outputter 2036 thatoutputs a scalable profile data 2054.

Decoders

FIG. 23 is a block diagram that depicts a decoder, according to oneembodiment of the present invention. According to one embodiment,decoder 2320 is any device, such as a portable communications device, astationary computing device, or a digital image capturing device, thatcan decode any type of scalable media, such as JPEG 2000 or MPEG.

Typically, users interact with a client device, such as a television, aPDA, a DVD player, or a computer monitor, to request that one or moreimages be displayed on the client device. A decoder 2320 may communicatewith or be a part of a client device 2380. As a part of providing theimage to the client device 2380 for viewing by a user, the decoder 2320and client 2380 may communicate about the client's 2380 capabilities fordisplaying the images and/or the users desire in how to view the images.As a part of this communication, the decoder 2320 may request thedesired scalability attribute 2372. For example, the desired scalabilityattribute 2372 may indicate the band width that the client 2380 capableof handling or whether the user want to see the images in black andwhite or in color. The client 2380 may return the desired scalableattribute 2374 to the decoder 2320. Other examples of client devicesinclude, but are not limited to, a portable communications device, astationary computing device and a digital image capturing device.

There are many legacy decoders that are not capable of decoding and/ordecrypting scalable media, such as JPEG 2000 files or MPEG files.According to one embodiment, decoder 2320 is a legacy device. Byinstalling software or components such as scalable media receiver 2330,scalable profile data receiver 2340, protection profile data receiver2350, and renderer 2362 onto a legacy decoder 2320, that legacy decoder2320 may become capable of decoding and/or decrypting scalable media2052, according to embodiments already described herein. For example,renderer 2362 may decode and/or decrypt scalable media 2052 to providerendered media data 2376 for viewing on client 2380.

As depicted in FIG. 23, scalable media receiver 2330 is configured toreceive scalable media 2052, scalable profile data 2054, and protectionprofile data 2056. According to one embodiment, the scalable mediareceiver 2330 includes receivers 2330, 2340, 2350 that are configured toreceive the scalable media 2052, scalable profile data 2054, andprotection profile data 2056. The receivers 2330, 2340, 2350 are coupledto renderer 2362, according to another embodiment. According toembodiments already described herein, renderer 2362 decodes scalablemedia 2052 using scalable profile data 2054 by extracting portions ofscalable media 2052 based on the scalable profile data 2054 and/ordecrypts scalable media 2052 using protection profile data 2056.

In still another embodiment, two or more of the receivers (2330, 2340,2350) may be combined. For example, scalable media receiver 2330 and thescalable profile data receiver 2340 may be combined into one receiver.Similarly, scalable media receiver 2330 and protection profile datareceiver 2350 may be combined into one receiver. Alternatively, allthree receivers (2330, 2340, 2350) may be combined into one receiver.

Scalable media 2052, scalable profile data 2054, and protection profiledata 2056 may be separate files or combined together in a single file inany combination, according to one embodiment, For example, scalablemedia 2052, scalable profile data 2054, and protection profile data 2056may be in a single file together. In a second example, scalable media2052 and scalable profile data 2054 may be in a file together, whileprotection profile data 2056 is in a separate file. In a third example,scalable media 2052 and protection profile data 2056 may be in a filetogether, while scalable profile data 2054 is in a separate file.

FIG. 24 is a block diagram that depicts a decoder, according to anotherembodiment of the present invention. FIG. 24 depicts a decoder 2420 thatis similar to the decoder 2320 depicted in FIG. 23, except that decoder2420 does not include a protection profile data receiver 2350 whichreceives protection profile data 2056. Therefore, according to oneembodiment, unlike renderer 2362 depicted in FIG. 23, renderer 2462 inFIG. 24 does not decrypt scalable media 2052 with protection profiledata 2350. However, in yet another embodiment, renderer 2462 may decryptscalable media 2052 using another technique.

FIG. 25 is a block diagram that depicts a decoder, according to yetanother embodiment of the present invention. FIG. 25 depicts a decoder2520 that does not include a scalable profile data receiver whichreceives scalable profile data 2054. Therefore, according to oneembodiment, renderer 2562 in FIG. 25 may include logic for parsingscalable media 2052 without a scalable profile data 2054. For example,assuming that scalable media 2052 is a JPEG 2000 file, renderer 2562 maybe able to read a table of contents for scalable media 2052 to determinewhat segments of scalable media 2052 correspond to segments referred toin the cryptographic mapping associated with protection profile data2056 in order to decrypt scalable media 2052. However, in yet anotherembodiment, renderer 2462 may decrypt scalable media 2052 using anothertechnique. In yet another embodiment, renderer 2462 may not be fullyaware of a table of contents associated with scalable media 2052, but,may only have a minimal amount of logic for parsing certain aspects ofthe table of contents.

According to one embodiment, any one of scalable media 2052, scalableprofile data 2054, and/or protection profile data 2056 may becommunicated to a decoder 2320, 2420, 2520 over a network. According toanother embodiment, any one of scalable media 2052, scalable profiledata 2054, and/or protection profile data 2056 may be retrieved bydecoder 2320, 2420, 2520 from a storage device. The storage device maybe a part of the decoder 2320, 2420, 2520.

According to one embodiment, decoders 2320, 2420, 2520 may transmitrendered media data 2376 over a network to a client 2380. In yet anotherembodiment, rendered media data 2376 may be stored on a storage devicewhere it is subsequently retrieved by a client 2380. The storage devicemay be a part of the decoder 2320, 2420, 2520.

The discussion of decoders 2320, 2420, 2520 in this section has assumedthat the decoders 2320, 2420, 2520 were at the end of a chain of devicesthat include encoders, transcoders, and possible other decoders beforedelivering an image to a client 2380. According to one embodiment,renderers 2362, 2462, 2562 may have logic for parsing enough of thetable of contents associated with a scalable media 2052 to decompressthe scalable media 2052 and provide an image to a client 2380.

Transcoders

There are legacy devices that are not capable of transcoding scalablemedia, such as JPEG 2000 files or MPEG files. In this section, many waysof upgrading legacy devices with various software, hardware, and/ormicrocode components so that the legacy devices are capable oftranscoding scalable media shall be discussed.

FIG. 26 is a block diagram that depicts a transcoding unit, according toone embodiment of the present invention. Transcoding unit 2620 may beany device, such as a portable communications device, a stationarycomputing device, or a digital image capturing device that can transcodeany type of scalable media, such as JPEG 2000 or MPEG. As with thepreviously described decoders, transcoding unit 2620 may request adesired scalable attribute 2372 from a device 2670, according to oneembodiment. In this case, device 2670 may provide the desired scalableattribute 2374. According to one embodiment, device 2670 may be anothertranscoding unit, a decoder, or a client device, among other things.

As depicted in FIG. 26, transcoding unit 2620 receives scalable media2052, scalable profile data 2054, and protection profile data 2056,according to one embodiment. Transcoding unit 2620, performs operationson scalable media 2052, scalable profile data 2054, and protectionprofile data 2056 to produce new scalable media 2662, new scalable mediadata 2664, and new protection profile data 2666, according to anotherembodiment. Further, transcoding unit 2620, according to yet anotherembodiment, includes a transcoder 2630 that extracts segments fromscalable media 2052 based on scalable profile data 2054, according toembodiments described here, and combines those segments to producetranscoded media 2640 that is communicated to an encoder 2650.

The encoder 2650, according to one embodiment, receives transcoded media2640, which may be scalable media or a non-scaled data such as a bitmap. Encoder 2650 may be a scalable media encoder, such as a JPEG 2000or an MPEG encoder, that is capable of taking a non-scaled bit map tocreate new scalable media 2662, according to one embodiment. In anotherembodiment, encoder 2650 may only have enough logic to generate scalablemedia 2662 for certain attributes.

Further, as depicted in FIG. 26, encoder 2650 includes a scalableprofile data generator 2654 and a protection profile data generator 2656that generate new scalable profile data 2664 and new protection profiledata 2666, according to one embodiment. In one embodiment, encoder 2650generates the scalable profile data 2664 and protection profile data2666 by analyzing the transcoded media 2640. In another embodiment,encoder 2650 may receive input, such as scalable attribute criteria 2014and/or protection attribute criteria 2016, as already described herein,to generate scalable profile data 2664 and protection profile data 2666.

In yet another embodiment, encoder 2650 produces new scalable profiledata 2664 by altering the offsets associated with the segments indicatedin the new scalable profile data 2054 to produce new scalable profiledata 2664 to reflect that certain segments have been extracted fromscalable media 2052, according to embodiments already described herein.In still another embodiment, encoder 2650 produces new scalable profiledata 2664 by marking what transcoding operations have been performed,e.g., what segments have been extracted, according to embodimentsalready described herein.

Similarly, according to another embodiment, encoder 2650 may produce newprotection profile data 2666 by modifying the cryptographic mappingassociated with the protection profile data 2056 to produce protectionprofile data 2666. In one embodiment, the cryptographic mapping may bemodified by altering offsets associated with the segments extracted fromscalable media 2052 on scalable profile data 2054, according toembodiments already described herein. In another embodiment, thecryptographic mapping may be modified by marking what transcodingoperations have been performed on scalable media 2052, according toembodiments already described herein.

FIG. 27 is a block diagram that depicts a transcoding unit, according toanother embodiment of the present invention. Transcoding unit 2720 maybe any device, such as a portable communications device, a stationarycomputing device, or a digital image capturing device that can transcodeany type of scalable media, such as JPEG 2000 or MPEG.

FIG. 27 depicts a transcoding unit 2720 that does not include a scalableprofile data receiver which receives scalable profile data 2054.Therefore, according to one embodiment, transcoder 2730 in FIG. 27 mayinclude logic for parsing scalable media 2052 without a scalable profiledata 2054. For example, assuming that scalable media 2052 is a JPEG 2000file, transcoder 2730 may be able to read a table of contents forscalable media 2052 to determine what segments of scalable media 2052correspond to segments referred to in the cryptographic mappingassociated with protection profile data 2056 in order to transcodescalable media 2052 and produce transcoded media 2640. Transcoder 2730may only have a minimal amount of logic for parsing certain aspects ofthe table of contents.

Transcoding unit 2720 further includes, an encoder 2750, according toone embodiment, that receives transcoded media 2640, which may bescalable media or non-scaled data such as a bit map. Encoder 2750 may bea scalable media encoder, such as a JPEG 2000 or an MPEG encoder, thatis capable of taking a non-scaled bit map to create new scalable media2662, according to one embodiment. In another embodiment, encoder 2750may only have enough logic to generate scalable media 2662 for certainattributes.

As depicted in FIG. 27, encoder 2750 includes a protection profile datagenerator 2656 that generates the new protection profile data 2666,according to one embodiment. In one embodiment, protection profile datagenerator 2656 generates the protection profile data 2666 by analyzingthe transcoded media 2640. In another embodiment, protection profiledata generator 2656 may receive input, such as protection attributecriteria 2016, as already described herein, to generate protectionprofile data 2666.

FIG. 28 is a block diagram that depicts a transcoding unit, according toanother embodiment of the present invention. Transcoding unit 2820 maybe any device, such as a portable communications device, a stationarycomputing device, or a digital image capturing device that can transcodeany type of scalable media, such as JPEG 2000 or MPEG.

According to one embodiment, transcoding units do not need theprotection profile data 2656 since, typically, transcoding units do notdecrypt the media data, such as scalable media data 2052, that thetranscoding units receive. For example, FIG. 28 depicts a transcodingunit 2820 that is similar to the transcoding unit 2620 depicted in FIG.26, except that transcoding unit 2820 does not include a protectionprofile data receiver 2626. Therefore, according to one embodiment,transcoder 2830 depicted in FIG. 28 does not require logic for parsingprotection profile data 2056.

Transcoding unit 2820 further includes, an encoder 2850, according toone embodiment, that receives transcoded media 2640, which may bescalable media or a non-scaled data such as a bit map. Encoder 2850 maybe a scalable media encoder, such as a JPEG 2000 or an MPEG encoder,that is capable of taking a non-scaled bit map to create new scalablemedia 2662, according to one embodiment. In another embodiment, encoder2750 may only have enough logic to generate scalable media 2662 forcertain attributes.

Encoder 2850 includes a scalable profile data generator 2654 thatgenerates the new scalable profile data 2664, according to oneembodiment. In one embodiment, scalable profile data generator 2654generates the scalable profile data 2664 by analyzing the transcodedmedia 2640. In another embodiment, scalable profile data generator 2654may receive input, such as scalable attribute criteria 2014, as alreadydescribed herein, to generate scalable profile data 2664.

FIG. 29 is a block diagram that depicts a transcoding unit, according toanother embodiment of the present invention. Transcoding unit 2920 maybe any device, such as a portable communications device, a stationarycomputing device, or a digital image capturing device that can transcodeany type of scalable media, such as JPEG 2000 or MPEG. As withtranscoding unit 2620, transcoding unit receives scalable media 2052,scalable profile data 2054, and protection profile data 2056 withrespective receivers (2622, 2624, 2626), according to one embodiment.Further transcoding unit 2920 outputs scalable media 2662, scalableprofile data j64, and protection profile data j66 with respectiveoutputters (2942, 2944, 2946), according to another embodiment.

As depicted in FIG. 29, according to one embodiment, transcoding unit2920 includes a transcoder 2930 that extracts segments from scalablemedia 2052 based on scalable profile data 2054, according to embodimentsdescribed here, to produce new scalable media 2662. Further, accordingto another embodiment, transcoder 2930 processes scalable profile data2054 to produce new scalable profile data 2664.

In one embodiment, transcoder 2930 produces new scalable profile data2664 by altering the offsets associated with the segments indicated inthe new scalable profile data 2054 to produce scalable profile data 2664to reflect that certain segments have been extracted from scalable media2052, according to embodiments already described herein. In anotherembodiment, transcoder 2930 produces new scalable profile data 2664 bymarking what transcoding operations have been performed, e.g., whatsegments have been extracted from scalable media 2052, according toembodiments already described herein.

Transcoder 2930 may produce new protection profile data 2666 bymodifying the cryptographic mapping associated with the protectionprofile data 2056 to produce protection profile data 2666. In oneembodiment, the cryptographic mapping may be modified by alteringoffsets associated with the segments extracted from scalable media 2052based on scalable profile data 2054, according to embodiments alreadydescribed herein. In another embodiment, the cryptographic mapping maybe modified by marking what transcoding operations have been performed,according to embodiments already described herein.

FIG. 30 is a block diagram that depicts a transcoding unit, according toanother embodiment of the present invention. Transcoding unit 3020 maybe any device, such as a portable communications device, a stationarycomputing device, or a digital image capturing device that can transcodeany type of scalable media, such as JPEG 2000 or MPEG.

FIG. 30 depicts a transcoding unit 3020 that does not include a scalableprofile data receiver which receives scalable profile data 2054.Therefore, according to one embodiment, transcoder 3030 in FIG. 30 mayinclude logic for parsing scalable media 2052 without a scalable profiledata 2054. For example, assuming that scalable media 2052 is a JPEG 2000file, transcoder 3030 may be able to read a table of contents forscalable media 2052 to determine what segments of scalable media 2052correspond to segments referred to in the cryptographic mappingassociated with protection profile data 2056 in order to modify thecryptographic mapping as already described herein. In yet anotherembodiment, transcoder 3030 may not be fully aware of a table ofcontents associated with scalable media 2052, but, may only have aminimal amount of logic for parsing certain aspects of the table ofcontents.

FIG. 31 is a block diagram that depicts a transcoding unit, according toanother embodiment of the present invention. FIG. 31 depicts atranscoding unit 3120 that is similar to the transcoding unit 2920depicted in FIG. 29, except that transcoding unit 3120 does not includea protection profile data receiver 2626. Therefore, according to oneembodiment, transcoder 2830 depicted in FIG. 28 does not require logicfor parsing protection profile data or for modifying the cryptographicmapping of a protection profile data, according to embodiments alreadydescribed herein.

FIG. 32 is a block diagram that depicts a transcoding unit, according toanother embodiment of the present invention. FIG. 32 depicts atranscoding unit 3220 that is similar to the transcoding unit 2920depicted in FIG. 29, except that transcoding unit 3220 includes atranscoder 3230 with a protection modifier 3232. According to oneembodiment, protection modifier 3232 may modify protection or add layersof protection, according to embodiments described herein. Although onlyFIG. 32 depicts a protection modifier 3232 , according to anotherembodiment, the transcoding units 2620, 2720, 2820, 2920, 3020, 3120,3220 (FIGS. 26-32) may also include protection modifiers.

In yet another embodiment, input, such as scalable attribute criteria2014 and/or protection attribute criteria 2016 may be received by anencoder 2650, 2750, 2850 (FIGS. 26-28) to generate scalable profile data2664 and/or protection profile data 2666.

According to one embodiment, the scalable profile data 2054 and/or theprotection profile data 2056 may be encrypted, in which case, whatevertranscoding unit 2620, 2720, 2820, 2920, 3020, 3120, 3220 (FIGS. 26-32)that receives the data (2054, 2056) would need the key to decrypt thedata (2054, 2056).

There are legacy transcoders that are not capable of transcodingscalable media, such as JPEG 2000 files or MPEG files. According to oneembodiment, transcoding units 2620, 2720, 2820, 2920, 3020, 3120, 3220(FIGS. 26-32) are legacy devices. By installing software, hardware,and/or microcode components such as receivers 2622, 2624, 2626,transcoders 2630, 2730, 2830, 2930, 3030, 3130, 3230 encoders 2650,2750, 2850, and outputters 2942, 2944, 2946 onto transcoding units 2620,2720, 2820, 2920, 3020, 3120, 3220 the transcoding units 2620, 2720,2820, 2920, 3020, 3120, 3220 may become capable of transcoding scalablemedia 2052, according to embodiments already described herein.

According to one embodiment, scalable media 2052 is encrypted. Forexample, scalable profile data 2054 enable transcoders 2630, 2830, 2930,3130, 3230 (FIGS. 26, 28, 29, 31, 32) to extract segments from scalablemedia 2052, according to embodiments already described herein. In orderto do this, the transcoders 2630, 2830, 2930, 3130, 3230 may receive thekeys to un-encrypt the scalable media 2052. According to anotherembodiment, scalable media 2052 is not encrypted.

Scalable media data 2052, scalable profile data 2054, and/or protectionprofile data 2056 may be communicated to transcoding units 2620, 2720,2820, 2920, 3020, 3120, 3220 (FIGS. 26-32) from another device, such asan encoder or another transcoder, using a network, according to oneembodiment. In another embodiment, scalable media data 2052, scalableprofile data 2054, and/or protection profile data 2056 may be retrievedby transcoding units 2620, 2720, 2820, 2920, 3020, 3120, 3220 from astorage device. In still another embodiment, the storage device may be apart of the respective transcoding unit 2620, 2720, 2820, 2920, 3020,3120, 3230.

Scalable media 2662, scalable profile data 2664, and/or protectionprofile data 2666 (FIGS. 26-32) may be transmitted to another device,such as another transcoder or a decoder, using a network, according toone embodiment. According to another embodiment, scalable media 2662,scalable profile data 2664, and/or protection profile data 2666 may bestored on a storage device where another device, such as a transcoder ora decoder may retrieve them. In still another embodiment, the storagedevice may be a part of the respective transcoding unit 2620, 2720,2820, 2920, 3020, 3120, 3220.

In yet another embodiment, two or more of the receivers 2622, 2624, 2626(FIGS. 26-32) may be combined into one receiver. For example, scalablemedia receiver 2622 and scalable profile data receiver 2624 may becombined into one receiver. Similarly, scalable media receiver 2622 andprotection profile data receiver 2626 may be combined into one receiver.Alternatively, all three receivers (2622, 2624, 2626) may be combinedinto one receiver.

In still another embodiment, two or more of the outputters 2942, 2944,2946 (FIGS. J29, 30, 31, 32) may be combined into one outputter. Forexample, scalable media outputter 2942 and scalable profile dataoutputter 2944 may be combined into one outputter. Similarly, scalablemedia outputter 2942 and protection profile data outputter 2946 may becombined into one outputter. Alternatively, all three outputters (2942,2944, 2946) may be combined into one outputter.

In yet another embodiment, the generators 2654, 2656 (FIGS. 29, 30, 31,32) may be combined into one generator.

As already discussed, scalable media 2052, scalable profile data 2054,and protection profile data 2056 (FIGS. 26-32) may be in separate filesor combined together in a single file in any combination, according toone embodiment.

According to another embodiment, to maintain the security of scalablemedia 2052, the transcoding unit 2620, 2720, 2820, 2920, 3020, 3120,3220 (FIGS. 26-32) does not receive the keys that were used forencrypting the scalable media 2052. In this case, only a decoder (2320,2420, 2520) would receive the keys in order to decrypt the scalablemedia 2052.

Encoders 2020, 2120, 2220 (FIGS. 20-22) and transcoding units 2620,2720, 2820, 2920, 3020, 3120, 3220 (FIGS. 26-32) may be in a singledevice, according to one embodiment. Encoders 2020, 2120, 2220 (FIGS.20-22) and decoders 2320, 2420, 2520 (FIGS. 23-25) may be in a singledevice, according to another embodiment. In yet another embodiment. Instill another embodiment, transcoding units 2620, 2720, 2820, 2920,3020, 3120, 3220 (FIGS. 26-32) and decoders 2320, 2420, 2520 (FIGS.23-32) may be in a single device. Encoders 2020, 2120, 2220 (FIGS.20-22), transcoding units 2620, 2720, 2820, 2920, 3020, 3120, 3220(FIGS. 26-32), and decoders 2320, 2420, 2520 (FIGS. 23-25) may all threebe in a single device.

The functions associated with the encoders 2020, 2120, 2220, transcodingunits 2620, 2720, 2820, 2920, 3020, 3120, 3220, and decoders 2320, 2420,2520 depicted in FIGS. 26-32, m may be moved around and combined in manyways that would be apparent to one of ordinary skill in the art.

Embodiments of the present invention are thus described. While thepresent invention has been described in particular embodiments, itshould be appreciated that the present invention should not be construedas limited by such embodiments, but rather construed according to thefollowing claims.

1. A media data decoding apparatus comprising: a scalable media receiverfor receiving scalable media encoded to a first encoding scheme; ascalable profile data receiver for receiving scalable profile data saidscalable profile data comprising a reference to segments of saidscalable media, said reference beyond the syntax of said encodingscheme, said reference for locating said segments without requiringknowledge of said encoding scheme; and a renderer that decodes saidscalable media and generates rendered media data for viewing on a clientdevice, wherein said rendered media data is rendered based, at least inpart, on said scalable media and said scalable profile data, such thatsaid scalable media data is decoded at said media data decoding devicefor viewing on said client device.
 2. The media data decoding apparatusof claim 1, wherein said media data decoding apparatus is a legacydecoder.
 3. The media data decoding apparatus of claim 1, wherein saidscalable profile data is encrypted.
 4. The media data decoding apparatusof claim 1, wherein said scalable profile data is unencrypted.
 5. Themedia data decoding apparatus of claim 1, wherein two or more of saidscalable media and said scalable profile data are received together in asingle file.
 6. The media data decoding apparatus of claim 5, whereinsaid scalable profile data and said scalable media are in said singlefile and said scalable profile data is at the beginning of said file. 7.The media data decoding apparatus of claim 5, wherein said scalableprofile data and said scalable media are in said single file and saidscalable profile data is not at the beginning of said file.
 8. The mediadata decoding apparatus of claim 1, wherein said scalable media decodingapparatus is an apparatus selected from the group consisting of aportable communications device, a stationary computing device, or adigital image capturing device.
 9. A media data decoding apparatuscomprising: a scalable media receiver for receiving scalable media, saidscalable media encoded according to an encoding scheme and associatedwith information referencing segments of said scalable media whereinsaid segments are combinable to produce a scaled version of saidscalable media, said referencing beyond the syntax of said encodingscheme; a protection profile data receiver for receiving protectionprofile data; and a renderer that decodes said scalable media andgenerates rendered media data for viewing on a client device, whereinsaid rendered media data is rendered based, at least in part, on saidscalable media and said protection profile data, such that said scalablemedia data is decoded at said media data decoding device for viewing onsaid client device.
 10. The media data decoding apparatus of claim 9,wherein said media data decoding apparatus is a legacy decoder.
 11. Themedia data decoding apparatus of claim 9, wherein said protectionprofile data is encrypted.
 12. The media data decoding apparatus ofclaim 9, wherein said protection profile data is unencrypted.
 13. Themedia data decoding apparatus of claim 9 wherein two or more of saidscalable media and said protection profile data are received together ina single file.
 14. The media data decoding apparatus of claim 13,wherein said protection profile data and said scalable media are in saidsingle file and said protection profile data is at the beginning of saidfile.
 15. The media data decoding apparatus of claim 13, wherein saidprotection profile data and said scalable media are in said single fileand said protection profile data is not at the beginning of said file.16. The media data decoding apparatus of claim 9, wherein said scalablemedia decoding apparatus is an apparatus selected from the groupconsisting of a portable communications device, a stationary computingdevice, or a digital image capturing device.
 17. A media data decodingapparatus comprising: a scalable media receiver for receiving scalablemedia encoded according to an encoding scheme; a protection profile datareceiver for receiving protection profile data; a scalable profile datareceiver for receiving scalable profile data said scalable profile datacomprising a reference to segments of said scalable media, saidreference beyond the syntax of said encoding scheme, said reference forlocating said segments without requiring knowledge of said encodingscheme; and a renderer that decodes said scalable media and generatesrendered media data for viewing on a client device, wherein saidrendered media data is rendered based, at least in part, on saidscalable media and said scalable profile data, said scalable profiledata, and said protection profile data, such that said scalable mediadata is decoded at said media data decoding device for viewing on saidclient device.
 18. The media data decoding apparatus of claim 17,wherein said media data decoding apparatus is a legacy decoder.
 19. Themedia data decoding apparatus of claim 17, wherein said scalable profiledata is encrypted.
 20. The media data decoding apparatus of claim 17,wherein said scalable profile data is unencrypted.
 21. The media datadecoding apparatus of claim 17, wherein two or more of said scalablemedia, said scalable profile data, and said protection profile data arereceived together in a single file.
 22. The media data decodingapparatus of claim 21, wherein said scalable profile data and saidscalable media are in said single file and said scalable profile data isat the beginning of said file.
 23. The media data decoding apparatus ofclaim 21, wherein said scalable profile data and said scalable media arein said single file and said scalable profile data is not at thebeginning of said file.
 24. The media data decoding apparatus of claim21, wherein said protection profile data and said scalable media are insaid single file and said protection profile data is at the beginning ofsaid file.
 25. The media data decoding apparatus of claim 21, whereinsaid protection profile data and said scalable media are in said singlefile and said protection profile data is not at the beginning of saidfile.
 26. The media data decoding apparatus of claim 17, wherein saidscalable media decoding apparatus is an apparatus selected from thegroup consisting of a portable communications device, a stationarycomputing device, or a digital image capturing device.